[PATCH v2][Kimchi 0/4] Support to edit max memory
V2: - Includes fixes from reviews; - New validate_memory function to be used in Templates and Guests creation/update processes; - Adds new function to update objectstore, to update 'memory' from old to new schema. V1: This patchset implements the support in backend to allow user to change maxMemory xml tag. It also changes the memory API from 'memory: XXX' to 'memory': {'current': XXX, 'maxmemory': YYY} maxMemory xml tag is not created by default anymore, when a guest is created. Tag will be created only when the user changes maxmemory manually, and if the value is different (higher) of 'current' memory. So in order to performe memory hotplug, an user has to stop guest, modify maxmemory and then start the guest. Tests were fixed and new ones were included. Rodrigo Trujillo (4): Add support to edit max memory in Templates Add function to update 'memory' in objectstore Add support to edit max memory in Guest Changes and add tests to support new memory API (templates/guest) API.json | 41 ++++---- docs/API.md | 29 ++++-- i18n.py | 12 ++- model/templates.py | 64 +++++++++++- model/vms.py | 245 ++++++++++++++++++++++++-------------------- osinfo.py | 16 ++- root.py | 4 + template.conf | 10 +- tests/test_livemigration.py | 6 +- tests/test_model.py | 22 ++-- tests/test_rest.py | 23 ++++- tests/test_template.py | 38 +++++-- tests/test_vmtemplate.py | 26 +++-- utils.py | 46 +++++++++ vmtemplate.py | 75 ++++++-------- 15 files changed, 428 insertions(+), 229 deletions(-) -- 2.1.0
This patch changes the 'memory' parameter in API of Templates to: memory: {current: XXX, maxmemory: YYY} Other changes include: * enable maxmemory edition * remove max memory tests and limits * keeps max memory limit to 1TiB * changes templates.conf to suport max memory * set default memory and maxmemory to 1024 Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo@linux.vnet.ibm.com> --- API.json | 34 ++++++++++++++++--------- docs/API.md | 18 ++++++++++--- i18n.py | 9 ++++--- model/templates.py | 64 +++++++++++++++++++++++++++++++++++++++++++--- model/vms.py | 5 ++-- osinfo.py | 16 ++++++++---- template.conf | 10 +++++--- vmtemplate.py | 75 ++++++++++++++++++++++-------------------------------- 8 files changed, 154 insertions(+), 77 deletions(-) diff --git a/API.json b/API.json index 294be64..f9a4ced 100644 --- a/API.json +++ b/API.json @@ -65,6 +65,26 @@ } } } + }, + "memory": { + "description": "Current memory and maximum memory values", + "type": "object", + "properties": { + "current": { + "description": "Memory (MB) for the template", + "type": "integer", + "minimum": 512, + "error": "KCHTMPL0013E" + }, + "maxmemory": { + "description": "Maximum memory (MB) for the template", + "type": "integer", + "minimum": 512, + "error": "KCHTMPL0013E" + } + }, + "additionalProperties": false, + "error": "KCHTMPL0030E" } }, "properties": { @@ -452,12 +472,7 @@ "minLength": 1, "error": "KCHTMPL0011E" }, - "memory": { - "description": "Memory (MB) for the template", - "type": "integer", - "minimum": 512, - "error": "KCHTMPL0013E" - }, + "memory": { "$ref": "#/kimchitype/memory" }, "cdrom": { "description": "Path for cdrom", "type": "string", @@ -629,12 +644,7 @@ "minLength": 1, "error": "KCHTMPL0011E" }, - "memory": { - "description": "Memory (MB) for the template", - "type": "integer", - "minimum": 512, - "error": "KCHTMPL0013E" - }, + "memory": { "$ref": "#/kimchitype/memory" }, "cdrom": { "description": "Path for cdrom", "type": "string", diff --git a/docs/API.md b/docs/API.md index a46e80e..33d51ac 100644 --- a/docs/API.md +++ b/docs/API.md @@ -292,8 +292,11 @@ Represents a snapshot of the Virtual Machine's primary monitor. * name: The name of the Template. Used to identify the Template in this API * os_distro *(optional)*: The operating system distribution * os_version *(optional)*: The version of the operating system distribution - * memory *(optional)*: The amount of memory assigned to the VM. - Default is 1024M. + * memory *(optional)*: The memory parameters of the template, specify one + or both. Default values are 1024MiB: + * current: The amount of memory that will be assigned to the VM. + * maxmemory: The maximum total of memory that the VM can have. Amount + over current will be used exclusively for memory hotplug * cdrom *(optional)*: A volume name or URI to an ISO image. * networks *(optional)*: list of networks will be assigned to the new VM. Default is '[default]' @@ -396,7 +399,11 @@ A interface represents available network interface on VM. * icon: A URI to a PNG image representing this template * os_distro: The operating system distribution * os_version: The version of the operating system distribution - * memory: The amount of memory assigned to the VM in the unit of MB + * memory: The memory parameters of the template, that will be assigned to + the VM in the unit of MiB. + * current: The amount of memory that will be assigned to the VM. + * maxmemory: The maximum total of memory that the VM can have. Amount + over current will be used exclusively for memory hotplug * cdrom: A volume name or URI to an ISO image * storagepool: URI of the storagepool where template allocates vm storage. * networks *(optional)*: list of networks will be assigned to the new VM. @@ -439,7 +446,10 @@ A interface represents available network interface on VM. * icon: A URI to a PNG image representing this template * os_distro: The operating system distribution * os_version: The version of the operating system distribution - * memory: The amount of memory assigned to the VM + * memory: The memory parameters of the template, specify one or both of: + * current: The amount of memory that will be assigned to the VM. + * maxmemory: The maximum total of memory that the VM can have. Amount + over current will be used exclusively for memory hotplug * cdrom: A volume name or URI to an ISO image * networks *(optional)*: list of networks will be assigned to the new VM. * disks: An array of requested disks with the following optional fields diff --git a/i18n.py b/i18n.py index 59e75f7..36d4310 100644 --- a/i18n.py +++ b/i18n.py @@ -127,12 +127,13 @@ messages = { "KCHVM0068E": _("Unable to setup password-less login at remote host %(host)s using user %(user)s. Error: %(error)s"), "KCHVM0069E": _("Password field must be a string."), "KCHVM0070E": _("Error creating local host ssh rsa key of user 'root'."), - "KCHVM0071E": _("Memory value %(mem)s must be aligned to %(alignment)sMiB."), + "KCHVM0071E": _("%(param)s value (%(mem)sMiB) must be aligned to %(alignment)sMiB."), "KCHVM0073E": _("Unable to update the following parameters while the VM is offline: %(params)s"), "KCHVM0074E": _("Unable to update the following parameters while the VM is online: %(params)s"), - "KCHVM0076E": _("VM %(name)s must have serial and console defined to open a web serial console"), "KCHVM0077E": _("Impossible to get the serial console of %(name)s"), + "KCHVM0078E": _("Memory or Maximum Memory value is higher than amount supported by the host: %(memHost)sMiB."), + "KCHVM0079E": _("Memory or Maximum Memory value is higher than maximum amount recommended: 1TiB"), "KCHVMHDEV0001E": _("VM %(vmid)s does not contain directly assigned host device %(dev_name)s."), "KCHVMHDEV0002E": _("The host device %(dev_name)s is not allowed to directly assign to VM."), @@ -165,7 +166,7 @@ messages = { "KCHTMPL0010E": _("Template distribution must be a string"), "KCHTMPL0011E": _("Template distribution version must be a string"), "KCHTMPL0012E": _("The number of CPUs must be an integer greater than 0"), - "KCHTMPL0013E": _("Amount of memory (MB) must be an integer greater than 512"), + "KCHTMPL0013E": _("Amount of memory and maximum memory (MB) must be an integer greater than 512"), "KCHTMPL0014E": _("Template CDROM must be a local or remote ISO file"), "KCHTMPL0015E": _("Invalid storage pool URI %(value)s specified for template"), "KCHTMPL0016E": _("Specify an ISO image as CDROM or a base image to create a template"), @@ -181,6 +182,8 @@ messages = { "KCHTMPL0027E": _("Invalid disk image format. Valid formats: bochs, cloop, cow, dmg, qcow, qcow2, qed, raw, vmdk, vpc."), "KCHTMPL0028E": _("When setting template disks, following parameters are required: 'index', 'pool name', 'format', 'size' or 'volume' (for scsi/iscsi pools)"), "KCHTMPL0029E": _("Disk format must be 'raw', for logical, iscsi, and scsi pools."), + "KCHTMPL0030E": _("Memory expects an object with one or both parameters: 'current' and 'maxmemory'"), + "KCHTMPL0031E": _("Memory value (%(mem)sMiB) must be equal or lesser than maximum memory value (%(maxmem)sMiB)"), "KCHPOOL0001E": _("Storage pool %(name)s already exists"), "KCHPOOL0002E": _("Storage pool %(name)s does not exist"), diff --git a/model/templates.py b/model/templates.py index 8a29e02..6aa4593 100644 --- a/model/templates.py +++ b/model/templates.py @@ -20,6 +20,8 @@ import copy import libvirt import os +import platform +import psutil import stat from wok.exception import InvalidOperation, InvalidParameter @@ -34,6 +36,12 @@ from wok.plugins.kimchi.utils import pool_name_from_uri from wok.plugins.kimchi.vmtemplate import VMTemplate +# In PowerPC, memories must be aligned to 256 MiB +PPC_MEM_ALIGN = 256 +# Max memory 1TB, in KiB +MAX_MEM_LIM = 1073741824 + + class TemplatesModel(object): def __init__(self, **kargs): self.objstore = kargs['objstore'] @@ -69,10 +77,8 @@ class TemplatesModel(object): # Validate cpu info t.cpuinfo_validate() - # Validate max memory - maxMem = (t._get_max_memory(t.info.get('memory')) >> 10) - if t.info.get('memory') > maxMem: - raise OperationFailed("KCHVM0041E", {'maxmem': str(maxMem)}) + # Validate memory + t._validate_memory() # Validate volumes for disk in t.info.get('disks'): @@ -180,6 +186,13 @@ class TemplateModel(object): cpu_info.update(new_cpu_info) params['cpu_info'] = cpu_info + # Fix memory values, because method update does not work recursively + new_mem = params.get('memory') + if new_mem is not None: + params['memory'] = copy.copy(old_t.get('memory')) + params['memory'].update(new_mem) + validate_memory(params['memory']) + new_t.update(params) for net_name in params.get(u'networks', []): @@ -199,12 +212,55 @@ class TemplateModel(object): return ident +def validate_memory(memory): + # + # All checking are made in Mib, so, expects memory values in Mib + # + current = memory.get('current') + maxmem = memory.get('maxmemory') + + # Check Host Memory + if hasattr(psutil, 'virtual_memory'): + host_memory = psutil.virtual_memory().total >> 10 >> 10 + else: + host_memory = psutil.TOTAL_PHYMEM >> 10 >> 10 + + # Memories must be lesser than 1TB and the Host memory limit + if (current > (MAX_MEM_LIM >> 10)) or (maxmem > (MAX_MEM_LIM >> 10)): + raise InvalidParameter("KCHVM0079E") + if (current > host_memory) or (maxmem > host_memory): + raise InvalidParameter("KCHVM0078E", {'memHost': host_memory}) + + # Current memory cannot be greater than maxMemory + if current > maxmem: + raise InvalidParameter("KCHTMPL0031E", + {'mem': str(current), + 'maxmem': str(maxmem)}) + + # make sure memory and Maxmemory are alingned in 256MiB in PowerPC + distro, _, _ = platform.linux_distribution() + if distro == "IBM_PowerKVM": + if current % PPC_MEM_ALIGN != 0: + raise InvalidParameter('KCHVM0071E', + {'param': "Memory", + 'mem': str(current), + 'alignment': str(PPC_MEM_ALIGN)}) + elif maxmem % PPC_MEM_ALIGN != 0: + raise InvalidParameter('KCHVM0071E', + {'param': "Maximum Memory", + 'mem': str(maxmem), + 'alignment': str(PPC_MEM_ALIGN)}) + + class LibvirtVMTemplate(VMTemplate): def __init__(self, args, scan=False, conn=None): self.conn = conn VMTemplate.__init__(self, args, scan) self.set_cpu_info() + def _validate_memory(self): + validate_memory(self.info['memory']) + def cpuinfo_validate(self): cpu_model = CPUInfoModel(conn=self.conn) diff --git a/model/vms.py b/model/vms.py index 23e0df9..5fb27d3 100644 --- a/model/vms.py +++ b/model/vms.py @@ -53,13 +53,13 @@ from wok.plugins.kimchi.model.config import CapabilitiesModel from wok.plugins.kimchi.model.cpuinfo import CPUInfoModel from wok.plugins.kimchi.model.featuretests import FeatureTests from wok.plugins.kimchi.model.templates import TemplateModel +from wok.plugins.kimchi.model.templates import MAX_MEM_LIM, PPC_MEM_ALIGN from wok.plugins.kimchi.model.utils import get_ascii_nonascii_name, get_vm_name from wok.plugins.kimchi.model.utils import get_metadata_node from wok.plugins.kimchi.model.utils import remove_metadata_node from wok.plugins.kimchi.model.utils import set_metadata_node from wok.plugins.kimchi.screenshot import VMScreenshot from wok.plugins.kimchi.utils import template_name_from_uri -from wok.plugins.kimchi.vmtemplate import MAX_MEM_LIM, PPC_MEM_ALIGN from wok.plugins.kimchi.xmlutils.cpu import get_cpu_xml, get_numa_xml from wok.plugins.kimchi.xmlutils.cpu import get_topology_xml from wok.plugins.kimchi.xmlutils.disk import get_vm_disk_info, get_vm_disks @@ -256,7 +256,8 @@ class VMModel(object): if 'memory' in params and distro == "IBM_PowerKVM": if params['memory'] % PPC_MEM_ALIGN != 0: raise InvalidParameter('KCHVM0071E', - {'mem': str(params['memory']), + {'param': "Memory", + 'mem': str(params['memory']), 'alignment': str(PPC_MEM_ALIGN)}) dom = self.get_vm(name, self.conn) diff --git a/osinfo.py b/osinfo.py index 2ec5c3e..8102548 100644 --- a/osinfo.py +++ b/osinfo.py @@ -110,11 +110,13 @@ def _get_tmpl_defaults(): ConfigObj returns a dict like below when no changes were made in the template configuration file (template.conf) - {'main': {}, 'storage': {'disk.0': {}}, 'processor': {}, 'graphics': {}} + {'main': {}, 'memory': {}, 'storage': {'disk.0': {}}, 'processor': {}, + 'graphics': {}} The default values should be like below: - {'main': {'networks': ['default'], 'memory': '1024'}, + {'main': {'networks': ['default']}, + 'memory': {'current': 1024, 'maxmemory': 1024}, 'storage': { 'disk.0': {'format': 'qcow2', 'size': '10', 'pool': '/plugins/kimchi/storagepools/default'}}, 'processor': {'vcpus': '1', 'maxvcpus': 1}, @@ -123,7 +125,8 @@ def _get_tmpl_defaults(): # Create dict with default values tmpl_defaults = defaultdict(dict) tmpl_defaults['main']['networks'] = ['default'] - tmpl_defaults['main']['memory'] = _get_default_template_mem() + tmpl_defaults['memory'] = {'current': _get_default_template_mem(), + 'maxmemory': _get_default_template_mem()} tmpl_defaults['storage']['disk.0'] = {'size': 10, 'format': 'qcow2', 'pool': 'default'} tmpl_defaults['processor']['vcpus'] = 1 @@ -145,8 +148,11 @@ def _get_tmpl_defaults(): 'cdrom_bus': 'ide', 'cdrom_index': 2, 'mouse_bus': 'ps2'} # Parse main section to get networks and memory values - main_section = default_config.pop('main') - defaults.update(main_section) + defaults.update(default_config.pop('main')) + defaults['memory'] = default_config.pop('memory') + + defaults['memory']['current'] = int(defaults['memory']['current']) + defaults['memory']['maxmemory'] = int(defaults['memory']['maxmemory']) # Parse storage section to get disks values storage_section = default_config.pop('storage') diff --git a/template.conf b/template.conf index 3839be4..c4598f1 100644 --- a/template.conf +++ b/template.conf @@ -3,13 +3,17 @@ # [main] -# Memory in MB -#memory = 1024 - # List of networks separated by comma # Represents the virtual network interfaces to be assigned to guest #networks = default, +[memory] +# Memory in MB +# current = 1024 + +# Maximum value of memory to be assigned to guest in MB +# maxmemory = 1024 + [storage] # Specify multiple [[disk.X]] sub-sections to add multiples disks to guest diff --git a/vmtemplate.py b/vmtemplate.py index ef17ff6..4757d9f 100644 --- a/vmtemplate.py +++ b/vmtemplate.py @@ -19,7 +19,6 @@ import os import platform -import psutil import stat import time import urlparse @@ -42,12 +41,6 @@ from wok.plugins.kimchi.xmlutils.qemucmdline import get_qemucmdline_xml from wok.plugins.kimchi.xmlutils.serial import get_serial_xml -# In PowerPC, memories must be aligned to 256 MiB -PPC_MEM_ALIGN = 256 -# Max memory 1TB, in KiB -MAX_MEM_LIM = 1073741824 - - class VMTemplate(object): def __init__(self, args, scan=False): """ @@ -84,6 +77,14 @@ class VMTemplate(object): args['graphics'] = graphics default_disk = self.info['disks'][0] + + # Complete memory args, because dict method update is not recursive + if 'memory' in args: + if 'current' not in args['memory']: + args['memory']['current'] = self.info['memory']['current'] + if 'maxmemory' not in args['memory']: + args['memory']['maxmemory'] = self.info['memory']['maxmemory'] + # Override template values according to 'args' self.info.update(args) disks = self.info.get('disks') @@ -325,29 +326,8 @@ class VMTemplate(object): def _get_cpu_xml(self): # Include CPU topology, if provided cpu_topo = self.info.get('cpu_info', {}).get('topology', {}) - - return get_cpu_xml(0, self.info.get('memory') << 10, cpu_topo) - - def _get_max_memory(self, guest_memory): - # Setting maxMemory of the VM, which will be lesser value between: - # 1TB, (Template Memory * 4), Host Physical Memory. - max_memory = MAX_MEM_LIM - if hasattr(psutil, 'virtual_memory'): - host_memory = psutil.virtual_memory().total >> 10 - else: - host_memory = psutil.TOTAL_PHYMEM >> 10 - if host_memory < max_memory: - max_memory = host_memory - if (((guest_memory * 4) << 10) < max_memory): - max_memory = (guest_memory * 4) << 10 - - # set up arch to ppc64 instead of ppc64le due to libvirt compatibility - if self.info["arch"] == "ppc64": - # in Power, memory must be aligned in 256MiB - if (max_memory >> 10) % PPC_MEM_ALIGN != 0: - alignment = max_memory % (PPC_MEM_ALIGN << 10) - max_memory -= alignment - return max_memory + return get_cpu_xml(0, (self.info.get('memory').get('current')) << 10, + cpu_topo) def to_vm_xml(self, vm_name, vm_uuid, **kwargs): params = dict(self.info) @@ -375,27 +355,33 @@ class VMTemplate(object): else: params['cdroms'] = cdrom_xml - # max memory - params['max_memory'] = self._get_max_memory(params['memory']) - # Setting maximum number of slots to avoid errors when hotplug memory # Number of slots are the numbers of chunks of 1GB that fit inside # the max_memory of the host minus memory assigned to the VM. It # cannot have more than 32 slots in Power. - params['slots'] = ((params['max_memory'] >> 10) - - params['memory']) >> 10 - if params['slots'] < 0: + memory = self.info['memory'].get('current') + maxmemory = self.info['memory'].get('maxmemory') + + slots = (maxmemory - memory) >> 10 + if slots < 0: raise OperationFailed("KCHVM0041E", - {'maxmem': str(params['max_memory'] >> 10)}) - elif params['slots'] == 0: - params['slots'] = 1 - elif params['slots'] > 32: + {'maxmem': str(maxmemory)}) + elif slots == 0: + slots = 1 + elif slots > 32: distro, _, _ = platform.linux_distribution() if distro == "IBM_PowerKVM": - params['slots'] = 32 + slots = 32 + + # Rearrange memory parameters + params['memory'] = self.info['memory'].get('current') + params['max_memory'] = "" + if memory != maxmemory: + maxmem_xml = "<maxMemory slots='%s' unit='MiB'>%s</maxMemory>" + params['max_memory'] = maxmem_xml % (slots, maxmemory) # set a hard limit using max_memory + 1GiB - params['hard_limit'] = params['max_memory'] + (1024 << 10) + params['hard_limit'] = maxmemory + 1024 # vcpu element cpus = params['cpu_info']['vcpus'] @@ -411,9 +397,9 @@ class VMTemplate(object): <name>%(name)s</name> <uuid>%(uuid)s</uuid> <memtune> - <hard_limit unit='KiB'>%(hard_limit)s</hard_limit> + <hard_limit unit='MiB'>%(hard_limit)s</hard_limit> </memtune> - <maxMemory slots='%(slots)s' unit='KiB'>%(max_memory)s</maxMemory> + %(max_memory)s <memory unit='MiB'>%(memory)s</memory> %(vcpus_xml)s %(cpu_info_xml)s @@ -452,6 +438,7 @@ class VMTemplate(object): self._network_validate() self._iso_validate() self.cpuinfo_validate() + self._validate_memory() def cpuinfo_validate(self): pass -- 2.1.0
This patch adds a new function to update objectstore. Changing 'memory': XXX by 'memory': {'current': XXX, 'maxmemory': YYY}. Then old templates will have new format, avoiding errors. The rule is use default value of maxmemory provided by osinfo, however, if memory['current'] is greater, then maxmemory = memory['current'] Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo@linux.vnet.ibm.com> --- root.py | 4 ++++ utils.py | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 50 insertions(+) diff --git a/root.py b/root.py index b5aa78b..56eda12 100644 --- a/root.py +++ b/root.py @@ -26,6 +26,7 @@ from wok.plugins.kimchi.i18n import messages from wok.plugins.kimchi.control import sub_nodes from wok.plugins.kimchi.model import model as kimchiModel from wok.plugins.kimchi.utils import upgrade_objectstore_data +from wok.plugins.kimchi.utils import upgrade_objectstore_memory from wok.plugins.kimchi.utils import upgrade_objectstore_template_disks from wok.root import WokRoot from wok.utils import upgrade_objectstore_schema @@ -74,5 +75,8 @@ class Kimchi(WokRoot): '/plugins/kimchi') upgrade_objectstore_template_disks(self.model.conn) + # Upgrade memory data, if necessary + upgrade_objectstore_memory() + def get_custom_conf(self): return config.KimchiConfig() diff --git a/utils.py b/utils.py index 5a3f209..132a3c6 100644 --- a/utils.py +++ b/utils.py @@ -29,6 +29,7 @@ from urlparse import urlparse from wok.exception import InvalidParameter, OperationFailed from wok.plugins.kimchi import config +from wok.plugins.kimchi.osinfo import get_template_default from wok.utils import wok_log from wok.xmlutils.utils import xpath_get_text @@ -178,3 +179,48 @@ def upgrade_objectstore_template_disks(libv_conn): if conn: conn.close() wok_log.info("%d 'template' entries upgraded in objectstore.", total) + + +def upgrade_objectstore_memory(): + """ + Upgrade the value of a given JSON's item of all Templates. + Changes 'memory': XXX by 'memory': {'current': XXXX, + 'maxmemory': XXXX} + """ + total = 0 + try: + conn = sqlite3.connect(config.get_object_store(), timeout=10) + cursor = conn.cursor() + sql = "SELECT id,json FROM objects WHERE type='template'" + cursor.execute(sql) + for row in cursor.fetchall(): + template = json.loads(row[1]) + + # Get memory info + memory = template['memory'] + # New memory is a dictionary with 'current' and 'maxmemory' + if type(memory) is not dict: + maxmem = get_template_default('modern', + 'memory').get('maxmemory') + if maxmem < memory: + maxmem = memory + template['memory'] = {'current': memory, + 'maxmemory': maxmem} + else: + continue + + sql = "UPDATE objects SET json=? WHERE id=?" + cursor.execute(sql, (json.dumps(template), row[0])) + conn.commit() + total += 1 + except sqlite3.Error, e: + if conn: + conn.rollback() + raise OperationFailed("KCHUTILS0006E") + wok_log.error("Error while upgrading objectstore data:", e.args[0]) + finally: + if conn: + conn.close() + if total > 0: + wok_log.info( + "%d 'template' memory entries upgraded in objectstore.", total) -- 2.1.0
This patch changes the backend in order to allow user to edit max memory tag. It changes the memory API to return and receive: 'memory': {'current': XXX, 'maxmemory': YYY}. Other changes include: - maxMemory xml tag is not created by default anymore; - maxMemory xml tag is only created if user passes maxmemory; - if memory and maxmemory are equal, than maxMemory tag is removed; - keeps the limit to max memory and memory to 1TiB; - adds tests to json and modify documentation; Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo@linux.vnet.ibm.com> --- API.json | 7 +- docs/API.md | 11 ++- i18n.py | 3 +- model/vms.py | 246 ++++++++++++++++++++++++++++++++--------------------------- 4 files changed, 146 insertions(+), 121 deletions(-) diff --git a/API.json b/API.json index f9a4ced..3c56c93 100644 --- a/API.json +++ b/API.json @@ -329,12 +329,7 @@ } }, "cpu_info": { "$ref": "#/kimchitype/cpu_info" }, - "memory": { - "description": "The new amount (MB) of memory for the VM", - "type": "integer", - "minimum": 512, - "error": "KCHTMPL0013E" - } + "memory": { "$ref": "#/kimchitype/memory" } }, "additionalProperties": false }, diff --git a/docs/API.md b/docs/API.md index 33d51ac..7cb7952 100644 --- a/docs/API.md +++ b/docs/API.md @@ -116,7 +116,10 @@ server. writes across all virtual disks (kb/s). * io_throughput_peak: The highest recent value of 'io_throughput'. * uuid: UUID of the VM. - * memory: The amount of memory assigned to the VM (in MB) + * memory: The memory parameters of the VM in the unit of MiB. + * current: The amount of memory that is assigned to the VM. + * maxmemory: The maximum total of memory that the VM can have. Amount + over current will be used exclusively for memory hotplug * cpu_info: CPU-specific information. * vcpus: The number of CPUs assigned to the VM * maxvcpus: The maximum number of CPUs that can be assigned to the VM @@ -148,8 +151,10 @@ server. * name: New name for this VM (only applied for shutoff VM) * users: New list of system users. * groups: New list of system groups. - * memory: New amount of memory (MB) for this VM (if VM is running, new - value will take effect in next reboot) + * memory: New memory parameters of the VM in the unit of MiB. + Provide one or both. + * current: New amount of memory that will be assigned to the VM. + * maxmemory: New maximum total of memory that the VM can have. * graphics: A dict to show detail of VM graphics. * passwd *(optional)*: console password. When omitted a random password willbe generated. diff --git a/i18n.py b/i18n.py index 36d4310..33b9c9d 100644 --- a/i18n.py +++ b/i18n.py @@ -100,7 +100,7 @@ messages = { "KCHVM0039E": _("Cannot resume VM '%(name)s' because it is not paused."), "KCHVM0040E": _("Unable to resume VM '%(name)s'. Details: %(err)s"), "KCHVM0041E": _("Memory assigned is higher then the maximum allowed in the host: %(maxmem)sMib."), - "KCHVM0042E": _("VM '%(name)s' does not support live memory update. Update the memory with the machine offline to enable this feature."), + "KCHVM0042E": _("Guest '%(name)s' does not support live memory update. Please, with the guest offline, set Maximum Memory with a value greater then Memory to enable this feature."), "KCHVM0043E": _("Only increase memory is allowed in active VMs"), "KCHVM0044E": _("For live memory update, new memory value must be equal old memory value plus multiples of 1024 Mib"), "KCHVM0045E": _("There are not enough free slots of 1024 Mib in the guest."), @@ -134,6 +134,7 @@ messages = { "KCHVM0077E": _("Impossible to get the serial console of %(name)s"), "KCHVM0078E": _("Memory or Maximum Memory value is higher than amount supported by the host: %(memHost)sMiB."), "KCHVM0079E": _("Memory or Maximum Memory value is higher than maximum amount recommended: 1TiB"), + "KCHVM0080E": _("Cannot update Maximum Memory when guest is running."), "KCHVMHDEV0001E": _("VM %(vmid)s does not contain directly assigned host device %(dev_name)s."), "KCHVMHDEV0002E": _("The host device %(dev_name)s is not allowed to directly assign to VM."), diff --git a/model/vms.py b/model/vms.py index 5fb27d3..5702f80 100644 --- a/model/vms.py +++ b/model/vms.py @@ -52,8 +52,8 @@ from wok.plugins.kimchi.kvmusertests import UserTests from wok.plugins.kimchi.model.config import CapabilitiesModel from wok.plugins.kimchi.model.cpuinfo import CPUInfoModel from wok.plugins.kimchi.model.featuretests import FeatureTests -from wok.plugins.kimchi.model.templates import TemplateModel -from wok.plugins.kimchi.model.templates import MAX_MEM_LIM, PPC_MEM_ALIGN +from wok.plugins.kimchi.model.templates import PPC_MEM_ALIGN +from wok.plugins.kimchi.model.templates import TemplateModel, validate_memory from wok.plugins.kimchi.model.utils import get_ascii_nonascii_name, get_vm_name from wok.plugins.kimchi.model.utils import get_metadata_node from wok.plugins.kimchi.model.utils import remove_metadata_node @@ -97,6 +97,7 @@ XPATH_NAME = './name' XPATH_NUMA_CELL = './cpu/numa/cell' XPATH_TOPOLOGY = './cpu/topology' XPATH_VCPU = './vcpu' +XPATH_MAX_MEMORY = './maxMemory' # key: VM name; value: lock object vm_locks = {} @@ -251,16 +252,12 @@ class VMModel(object): vm_locks[name] = lock with lock: - # make sure memory is alingned in 256MiB in PowerPC - distro, _, _ = platform.linux_distribution() - if 'memory' in params and distro == "IBM_PowerKVM": - if params['memory'] % PPC_MEM_ALIGN != 0: - raise InvalidParameter('KCHVM0071E', - {'param': "Memory", - 'mem': str(params['memory']), - 'alignment': str(PPC_MEM_ALIGN)}) - dom = self.get_vm(name, self.conn) + # You can only change <maxMemory> offline, updating guest XML + if ("memory" in params) and ('maxmemory' in params['memory']) and\ + (DOM_STATE_MAP[dom.info()[0]] != 'shutoff'): + raise InvalidParameter("KCHVM0080E") + if DOM_STATE_MAP[dom.info()[0]] == 'shutoff': ext_params = set(params.keys()) - set(VM_OFFLINE_UPDATE_PARAMS) if len(ext_params) > 0: @@ -725,14 +722,14 @@ class VMModel(object): def _static_vm_update(self, vm_name, dom, params): old_xml = new_xml = dom.XMLDesc(0) - params = copy.deepcopy(params) # Update name name = params.get('name') nonascii_name = None + state = DOM_STATE_MAP[dom.info()[0]] + if name is not None: - state = DOM_STATE_MAP[dom.info()[0]] if state != 'shutoff': msg_args = {'name': vm_name, 'new_name': params['name']} raise InvalidParameter("KCHVM0003E", msg_args) @@ -771,7 +768,8 @@ class VMModel(object): new_xml = xml_item_remove(new_xml, XPATH_TOPOLOGY) # Updating memory and NUMA if necessary, if vm is offline - if 'memory' in params and not dom.isActive(): + if (not dom.isActive() and 'memory' in params and + params['memory'] != {}): new_xml = self._update_memory_config(new_xml, params) if 'graphics' in params: @@ -811,103 +809,113 @@ class VMModel(object): return (nonascii_name if nonascii_name is not None else vm_name, dom) def _update_memory_config(self, xml, params): - # Checks if NUMA memory is already configured, if not, checks if CPU - # element is already configured (topology). Then add NUMA element as - # apropriated root = ET.fromstring(xml) - numa_mem = xpath_get_text(xml, XPATH_NUMA_CELL + '/@memory') - maxvcpus = params.get('cpu_info', {}).get('maxvcpus') - if numa_mem == []: - if maxvcpus is None: - maxvcpus = int(xpath_get_text(xml, XPATH_VCPU)[0]) - cpu = root.find('./cpu') - if cpu is None: - cpu = get_cpu_xml(0, params['memory'] << 10) - root.insert(0, ET.fromstring(cpu)) - else: - numa_element = get_numa_xml(0, params['memory'] << 10) - cpu.insert(0, ET.fromstring(numa_element)) + # MiB to KiB + hasMem = 'current' in params['memory'] + hasMaxMem = 'maxmemory' in params['memory'] + oldMem = int(xpath_get_text(xml, XPATH_DOMAIN_MEMORY)[0]) >> 10 + maxMemTag = root.find(XPATH_MAX_MEMORY) + if maxMemTag is not None: + oldMaxMem = int(xpath_get_text(xml, XPATH_MAX_MEMORY)[0]) >> 10 else: - if maxvcpus is not None: - xml = xml_item_update( - xml, - XPATH_NUMA_CELL, - value='0', - attr='cpus') - root = ET.fromstring(xml_item_update(xml, XPATH_NUMA_CELL, - str(params['memory'] << 10), - attr='memory')) - - # Remove currentMemory, automatically set later by libvirt - currentMem = root.find('.currentMemory') - if currentMem is not None: - root.remove(currentMem) - - memory = root.find('.memory') - # Update/Adds maxMemory accordingly - if not self.caps.mem_hotplug_support: - if memory is not None: - memory.text = str(params['memory'] << 10) + oldMaxMem = oldMem + newMem = (params['memory'].get('current', oldMem)) << 10 + newMaxMem = (params['memory'].get('maxmemory', oldMaxMem)) << 10 + + validate_memory({'current': newMem >> 10, + 'maxmemory': newMaxMem >> 10}) + + def _get_slots(mem, maxMem): + slots = (maxMem - mem) >> 10 >> 10 + # Libvirt does not accepts slots <= 1 + if slots < 0: + raise InvalidParameter("KCHTMPL0031E", + {'mem': str(mem >> 10), + 'maxmem': str(maxMem >> 10)}) + elif slots == 0: + slots = 1 + + # max 32 slots on Power + distro, _, _ = platform.linux_distribution() + if distro == "IBM_PowerKVM" and slots > 32: + slots = 32 + return slots + # End of _get_slots + + # There is an issue in Libvirt/Qemu, where Guest does not start if + # memory and max memory are the same. So we decided to remove max + # memory and only add it if user explicitly provides it, willing to + # do memory hotplug + if hasMaxMem: + # Conditions: + if (maxMemTag is None) and (newMem != newMaxMem): + # Creates the maxMemory tag + max_mem_xml = E.maxMemory( + str(newMaxMem), + unit='Kib', + slots=str(_get_slots(newMem, newMaxMem))) + root.insert(0, max_mem_xml) + elif (maxMemTag is None) and (newMem == newMaxMem): + # Nothing to do + pass + elif (maxMemTag is not None) and (newMem != newMaxMem): + # Just update value in max memory tag + maxMemTag.text = str(newMaxMem) + maxMemTag.set('slots', str(_get_slots(newMem, newMaxMem))) + elif (maxMemTag is not None) and (newMem == newMaxMem): + # Remove the tag + root.remove(maxMemTag) + + # Update memory, if necessary + if hasMem: + # Remove currentMemory, automatically set later by libvirt, with + # memory value + currentMem = root.find('.currentMemory') + if currentMem is not None: + root.remove(currentMem) + + memory = root.find('.memory') + # If host/guest does not support memory hot plug, then there is + # NUMA configure, we must update the tag directly + if not self.caps.mem_hotplug_support: + if memory is not None: + memory.text = str(newMem) + else: + if memory is not None: + # Libvirt is going to set the value automatically with + # the value configured in NUMA tag + root.remove(memory) + + if (maxMemTag is not None) and (not hasMaxMem): + if newMem == newMaxMem: + root.remove(maxMemTag) + else: + maxMemTag.set('slots', str(_get_slots(newMem, newMaxMem))) + + # Set NUMA parameterers and create it if does not exist + # CPU tag DO NOT exist? Create it + cpu = root.find(XPATH_CPU) + if cpu is None: + cpu = get_cpu_xml(0, newMem) + root.insert(0, ET.fromstring(cpu)) else: - if memory is not None: - root.remove(memory) - - def _get_slots(maxMem): - slots = (maxMem - params['memory']) >> 10 - # Libvirt does not accepts slots <= 1 - if slots < 0: - raise OperationFailed("KCHVM0041E", - {'maxmem': str(maxMem)}) - elif slots == 0: - slots = 1 - - distro, _, _ = platform.linux_distribution() - if distro == "IBM_PowerKVM": - # max 32 slots on Power - if slots > 32: - slots = 32 - return slots - # End of _get_slots - - def _get_newMaxMem(): - # Setting max memory to 4x memory requested, host total memory, - # or 1 TB. This should avoid problems with live migration - newMaxMem = MAX_MEM_LIM - hostMem = self.conn.get().getInfo()[1] << 10 - if hostMem < newMaxMem: - newMaxMem = hostMem - mem = params.get('memory', 0) - if (mem != 0) and (((mem * 4) << 10) < newMaxMem): - newMaxMem = (mem * 4) << 10 - - distro, _, _ = platform.linux_distribution() - if distro == "IBM_PowerKVM": - # max memory 256MiB alignment - newMaxMem -= (newMaxMem % 256) - return newMaxMem - - maxMem = root.find('.maxMemory') - if maxMem is not None: - root.remove(maxMem) - - # Setting maxMemory - newMaxMem = _get_newMaxMem() - slots = _get_slots(newMaxMem >> 10) - max_mem_xml = E.maxMemory( - str(newMaxMem), - unit='Kib', - slots=str(slots)) - root.insert(0, max_mem_xml) - - # Setting memory hard limit to max_memory + 1GiB - memtune = root.find('memtune') - if memtune is not None: - hl = memtune.find('hard_limit') - if hl is not None: - memtune.remove(hl) - memtune.insert(0, E.hard_limit(str(newMaxMem + 1048576), - unit='Kib')) - + # Does NUMA tag exist ? + numaTag = cpu.find('./numa') + if numaTag is None: + numa_element = get_numa_xml(0, newMem) + cpu.insert(0, ET.fromstring(numa_element)) + else: + cellTag = cpu.find('./numa/cell') + cellTag.set('memory', str(newMem)) + + # Setting memory hard limit to max_memory + 1GiB + memtune = root.find('memtune') + if memtune is not None: + hl = memtune.find('hard_limit') + if hl is not None: + memtune.remove(hl) + memtune.insert(0, E.hard_limit(str(newMaxMem + 1048576), + unit='Kib')) return ET.tostring(root, encoding="utf-8") def _update_cpu_info(self, new_xml, dom, new_info): @@ -943,7 +951,8 @@ class VMModel(object): def _live_vm_update(self, dom, params): self._vm_update_access_metadata(dom, params) - if 'memory' in params and dom.isActive(): + if (('memory' in params) and ('current' in params['memory']) and + dom.isActive()): self._update_memory_live(dom, params) def _update_memory_live(self, dom, params): @@ -960,13 +969,21 @@ class VMModel(object): raise OperationFailed('KCHVM0042E', {'name': dom.name()}) # Memory live update must be done in chunks of 1024 Mib or 1Gib - new_mem = params['memory'] + new_mem = params['memory']['current'] old_mem = int(xpath_get_text(xml, XPATH_DOMAIN_MEMORY)[0]) >> 10 if new_mem < old_mem: raise OperationFailed('KCHVM0043E') if (new_mem - old_mem) % 1024 != 0: raise OperationFailed('KCHVM0044E') + # make sure memory is alingned in 256MiB in PowerPC + distro, _, _ = platform.linux_distribution() + if (distro == "IBM_PowerKVM" and new_mem % PPC_MEM_ALIGN != 0): + raise InvalidParameter('KCHVM0071E', + {'param': "Memory", + 'mem': str(new_mem), + 'alignment': str(PPC_MEM_ALIGN)}) + # Check slot spaces: total_slots = int(xpath_get_text(xml, './maxMemory/@slots')[0]) needed_slots = (new_mem - old_mem) >> 10 @@ -1190,11 +1207,18 @@ class VMModel(object): proc.join(1) self._serial_procs.remove(proc) + # Get max memory, or return "memory" if not set + maxmemory = xpath_get_text(xml, XPATH_MAX_MEMORY) + if len(maxmemory) > 0: + maxmemory = convert_data_size(maxmemory[0], 'KiB', 'MiB') + else: + maxmemory = memory + return {'name': name, 'state': state, 'stats': res, 'uuid': dom.UUIDString(), - 'memory': memory, + 'memory': {'current': memory, 'maxmemory': maxmemory}, 'cpu_info': cpu_info, 'screenshot': screenshot, 'icon': icon, -- 2.1.0
This patch fixes all tests, changing memory API, to support memory: {'current', 'maxmemory'}. It also include some new tests to test changing maxmemory parameter. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo@linux.vnet.ibm.com> --- tests/test_livemigration.py | 6 ++---- tests/test_model.py | 22 ++++++++++++++-------- tests/test_rest.py | 23 ++++++++++++++++++----- tests/test_template.py | 38 ++++++++++++++++++++++++++++++++------ tests/test_vmtemplate.py | 26 +++++++++++++++----------- 5 files changed, 81 insertions(+), 34 deletions(-) diff --git a/tests/test_livemigration.py b/tests/test_livemigration.py index 7119688..35d2175 100644 --- a/tests/test_livemigration.py +++ b/tests/test_livemigration.py @@ -98,14 +98,12 @@ class LiveMigrationTests(unittest.TestCase): params = {'name': u'template_test_vm_migrate', 'disks': [], 'cdrom': UBUNTU_ISO, - 'memory': 2048, - 'max_memory': 4096 << 10} + 'memory': {'current': 2048, 'maxmemory': 4096 << 10}} self.inst.templates_create(params) params = {'name': u'template_test_vm_migrate_nonshared', 'disks': [{'name': 'test_vm_migrate.img', 'size': 1}], 'cdrom': UBUNTU_ISO, - 'memory': 2048, - 'max_memory': 4096*1024} + 'memory': {'current': 2048, 'maxmemory': 4096*1024}} self.inst.templates_create(params) def tearDown(self): diff --git a/tests/test_model.py b/tests/test_model.py index 097c2d6..fcd1bbd 100644 --- a/tests/test_model.py +++ b/tests/test_model.py @@ -113,7 +113,7 @@ class ModelTests(unittest.TestCase): self.assertEquals(keys, set(info.keys())) self.assertEquals('running', info['state']) self.assertEquals('test', info['name']) - self.assertEquals(2048, info['memory']) + self.assertEquals(2048, info['memory']['current']) self.assertEquals(2, info['cpu_info']['vcpus']) self.assertEquals(2, info['cpu_info']['maxvcpus']) self.assertEquals(None, info['icon']) @@ -270,9 +270,10 @@ class ModelTests(unittest.TestCase): pool_uri = "/plugins/kimchi/storagepools/default" tmpl_info = {"cpu_info": {"vcpus": 1}, "name": tmpl_name, "graphics": {"type": "vnc", "listen": "127.0.0.1"}, - "networks": ["default"], "memory": 1024, "folder": [], - "icon": "images/icon-vm.png", "cdrom": "", - "os_distro": "unknown", "os_version": "unknown", + "networks": ["default"], "memory": {'current': 1024}, + "folder": [], "icon": "images/icon-vm.png", + "cdrom": "", "os_distro": "unknown", + "os_version": "unknown", "disks": [{"base": vol_path, "size": 10, "format": "qcow2", "pool": {"name": pool_uri}}]} @@ -755,7 +756,9 @@ class ModelTests(unittest.TestCase): def test_vm_memory_hotplug(self): config.set("authentication", "method", "pam") inst = model.Model(None, objstore_loc=self.tmp_store) - orig_params = {'name': 'test', 'memory': 1024, 'cdrom': UBUNTU_ISO} + orig_params = {'name': 'test', + 'memory': {'current': 1024, 'maxmemory': 3072}, + 'cdrom': UBUNTU_ISO} inst.templates_create(orig_params) with RollbackContext() as rollback: @@ -771,11 +774,12 @@ class ModelTests(unittest.TestCase): 'kimchi-vm1') # Hotplug memory, only available in Libvirt >= 1.2.14 - params = {'memory': 2048} + params = {'memory': {'current': 2048}} if inst.capabilities_lookup()['mem_hotplug_support']: inst.vm_update('kimchi-vm1', params) rollback.prependDefer(utils.rollback_wrapper, inst.vm_delete, 'kimchi-vm1') + params['memory']['maxmemory'] = 3072 self.assertEquals(params['memory'], inst.vm_lookup('kimchi-vm1')['memory']) else: @@ -789,7 +793,7 @@ class ModelTests(unittest.TestCase): # template disk format must be qcow2 because vmsnapshot # only supports this format - orig_params = {'name': 'test', 'memory': 1024, + orig_params = {'name': 'test', 'memory': {'current': 1024}, 'cpu_info': {'vcpus': 1}, 'cdrom': UBUNTU_ISO, 'disks': [{'size': 1, 'format': 'qcow2', 'pool': { @@ -907,7 +911,9 @@ class ModelTests(unittest.TestCase): self.assertEquals(4, vm_info['cpu_info']['maxvcpus']) # rename and increase memory when vm is not running - params = {'name': u'пeω-∨м', 'memory': 2048} + params = {'name': u'пeω-∨м', + 'memory': {'current': 2048, + 'maxmemory': 2048}} inst.vm_update('kimchi-vm1', params) rollback.prependDefer(utils.rollback_wrapper, inst.vm_delete, u'пeω-∨м') diff --git a/tests/test_rest.py b/tests/test_rest.py index 1fc0f71..8da7853 100644 --- a/tests/test_rest.py +++ b/tests/test_rest.py @@ -152,6 +152,19 @@ class RestTests(unittest.TestCase): resp = self.request('/plugins/kimchi/vms/vm-1', req, 'PUT') self.assertEquals(200, resp.status) + # Test max memory + req = json.dumps({'memory': {'maxmemory': 23}}) + resp = self.request('/plugins/kimchi/vms/vm-1', req, 'PUT') + self.assertEquals(400, resp.status) + + req = json.dumps({'memory': {'maxmemory': 'maxmem 80'}}) + resp = self.request('/plugins/kimchi/vms/vm-1', req, 'PUT') + self.assertEquals(400, resp.status) + + req = json.dumps({'memory': {'maxmemory': 3072}}) + resp = self.request('/plugins/kimchi/vms/vm-1', req, 'PUT') + self.assertEquals(200, resp.status) + resp = self.request('/plugins/kimchi/vms/vm-1/start', '{}', 'POST') self.assertEquals(200, resp.status) @@ -171,7 +184,7 @@ class RestTests(unittest.TestCase): # Check if there is support to memory hotplug, once vm is running resp = self.request('/plugins/kimchi/config/capabilities').read() conf = json.loads(resp) - req = json.dumps({'memory': 2048}) + req = json.dumps({'memory': {'current': 2048}}) resp = self.request('/plugins/kimchi/vms/vm-1', req, 'PUT') if conf['mem_hotplug_support']: self.assertEquals(200, resp.status) @@ -211,11 +224,11 @@ class RestTests(unittest.TestCase): resp = self.request('/plugins/kimchi/vms/vm-1', req, 'PUT') self.assertEquals(400, resp.status) - req = json.dumps({'memory': 100}) + req = json.dumps({'memory': {'current': 100}}) resp = self.request('/plugins/kimchi/vms/vm-1', req, 'PUT') self.assertEquals(400, resp.status) - req = json.dumps({'memory': 'ten gigas'}) + req = json.dumps({'memory': {'current': 'ten gigas'}}) resp = self.request('/plugins/kimchi/vms/vm-1', req, 'PUT') self.assertEquals(400, resp.status) @@ -225,7 +238,7 @@ class RestTests(unittest.TestCase): self.assertEquals(400, resp.status) params = {'name': u'∨м-црdαtеd', 'cpu_info': {'vcpus': 5}, - 'memory': 3072} + 'memory': {'current': 3072}} req = json.dumps(params) resp = self.request('/plugins/kimchi/vms/vm-1', req, 'PUT') self.assertEquals(303, resp.status) @@ -233,7 +246,7 @@ class RestTests(unittest.TestCase): self.request('/plugins/kimchi/vms/∨м-црdαtеd', req).read() ) # Memory was hot plugged - params['memory'] += 1024 + params['memory']['current'] += 1024 for key in params.keys(): self.assertEquals(params[key], vm[key]) diff --git a/tests/test_template.py b/tests/test_template.py index da0037e..08e92e4 100644 --- a/tests/test_template.py +++ b/tests/test_template.py @@ -70,7 +70,8 @@ class TemplateTests(unittest.TestCase): # Create a template without cdrom and disk specified fails with 400 t = {'name': 'test', 'os_distro': 'ImagineOS', - 'os_version': '1.0', 'memory': 1024, 'cpu_info': {'vcpus': 1}} + 'os_version': '1.0', 'memory': {'current': 1024}, + 'cpu_info': {'vcpus': 1}} req = json.dumps(t) resp = self.request('/plugins/kimchi/templates', req, 'POST') self.assertEquals(400, resp.status) @@ -146,10 +147,11 @@ class TemplateTests(unittest.TestCase): else: max_mem = (psutil.TOTAL_PHYMEM >> 10 >> 10) memory = max_mem + 1024 - t = {'name': 'test-maxmem', 'cdrom': '/tmp/mock.iso', 'memory': memory} + t = {'name': 'test-maxmem', 'cdrom': '/tmp/mock.iso', + 'memory': {'current': memory}} req = json.dumps(t) resp = self.request('/plugins/kimchi/templates', req, 'POST') - self.assertEquals(500, resp.status) + self.assertEquals(400, resp.status) self.assertTrue(str(max_mem) in resp.read()) def test_customized_tmpl(self): @@ -219,12 +221,36 @@ class TemplateTests(unittest.TestCase): update_tmpl = json.loads(resp.read()) self.assertEquals(update_tmpl['cpu_info'], cpu_info_data['cpu_info']) - # Update memory - req = json.dumps({'memory': 2048}) + # Test memory and max memory + # - memory greated than max memory (1024 default) + req = json.dumps({'memory': {'current': 2048}}) + resp = self.request(new_tmpl_uri, req, 'PUT') + self.assertEquals(400, resp.status) + # - max memory greater than 1TiB limit + req = json.dumps({'memory': {'maxmemory': 1073741824 + 1024}}) + resp = self.request(new_tmpl_uri, req, 'PUT') + self.assertEquals(400, resp.status) + self.assertTrue('KCHVM0079E' in resp.read()) + # - change only max memory + req = json.dumps({'memory': {'maxmemory': 3072}}) + resp = self.request(new_tmpl_uri, req, 'PUT') + self.assertEquals(200, resp.status) + update_tmpl = json.loads(resp.read()) + self.assertEquals(3072, update_tmpl['memory']['maxmemory']) + # - change only memory + req = json.dumps({'memory': {'current': 2048}}) + resp = self.request(new_tmpl_uri, req, 'PUT') + self.assertEquals(200, resp.status) + update_tmpl = json.loads(resp.read()) + self.assertEquals(2048, update_tmpl['memory']['current']) + self.assertEquals(3072, update_tmpl['memory']['maxmemory']) + # - change both values + req = json.dumps({'memory': {'current': 1024, 'maxmemory': 1024}}) resp = self.request(new_tmpl_uri, req, 'PUT') self.assertEquals(200, resp.status) update_tmpl = json.loads(resp.read()) - self.assertEquals(2048, update_tmpl['memory']) + self.assertEquals(1024, update_tmpl['memory']['current']) + self.assertEquals(1024, update_tmpl['memory']['maxmemory']) # Update cdrom cdrom_data = {'cdrom': '/tmp/mock2.iso'} diff --git a/tests/test_vmtemplate.py b/tests/test_vmtemplate.py index e157198..f5c44bd 100644 --- a/tests/test_vmtemplate.py +++ b/tests/test_vmtemplate.py @@ -98,21 +98,23 @@ class VMTemplateTests(unittest.TestCase): expr = "/domain/devices/graphics/@listen" self.assertEquals(graphics['listen'], xpath_get_text(xml, expr)[0]) expr = "/domain/maxMemory/@slots" - self.assertEquals('3', xpath_get_text(xml, expr)[0]) - expr = "/domain/maxMemory" - self.assertEquals(str((1024 * 4) << 10), xpath_get_text(xml, expr)[0]) + # The default is memory and maxmemory have the same value, so + # max memory tag is not set + self.assertEquals(0, len(xpath_get_text(xml, expr))) + expr = "/domain/memory" + self.assertEquals(str(1024), xpath_get_text(xml, expr)[0]) if hasattr(psutil, 'virtual_memory'): host_memory = psutil.virtual_memory().total >> 10 else: host_memory = psutil.TOTAL_PHYMEM >> 10 t = VMTemplate({'name': 'test-template', 'cdrom': self.iso, - 'memory': (host_memory >> 10) - 512}) - xml = t.to_vm_xml('test-vm', vm_uuid, graphics=graphics) - expr = "/domain/maxMemory" - self.assertEquals(str(host_memory), xpath_get_text(xml, expr)[0]) - expr = "/domain/maxMemory/@slots" - self.assertEquals('1', xpath_get_text(xml, expr)[0]) + 'memory': {'current': (host_memory >> 10) - 512}}) + try: + xml = t.to_vm_xml('test-vm', vm_uuid, graphics=graphics) + except Exception as e: + # Test current memory greater than maxmemory (1024/default) + self.assertTrue('KCHVM0041E' in e.message) def test_arg_merging(self): """ @@ -121,12 +123,14 @@ class VMTemplateTests(unittest.TestCase): """ graphics = {'type': 'vnc', 'listen': '127.0.0.1'} args = {'name': 'test', 'os_distro': 'opensuse', 'os_version': '12.3', - 'cpu_info': {'vcpus': 2, 'maxvcpus': 4}, 'memory': 2048, + 'cpu_info': {'vcpus': 2, 'maxvcpus': 4}, + 'memory': {'current': 2048, 'maxmemory': 3072}, 'networks': ['foo'], 'cdrom': self.iso, 'graphics': graphics} t = VMTemplate(args) self.assertEquals(2, t.info.get('cpu_info', {}).get('vcpus')) self.assertEquals(4, t.info.get('cpu_info', {}).get('maxvcpus')) - self.assertEquals(2048, t.info.get('memory')) + self.assertEquals(2048, t.info.get('memory').get('current')) + self.assertEquals(3072, t.info.get('memory').get('maxmemory')) self.assertEquals(['foo'], t.info.get('networks')) self.assertEquals(self.iso, t.info.get('cdrom')) self.assertEquals(graphics, t.info.get('graphics')) -- 2.1.0
Reviewed-by: Aline Manera <alinefm@linux.vnet.ibm.com> On 02/21/2016 01:14 AM, Rodrigo Trujillo wrote:
V2: - Includes fixes from reviews; - New validate_memory function to be used in Templates and Guests creation/update processes; - Adds new function to update objectstore, to update 'memory' from old to new schema.
V1: This patchset implements the support in backend to allow user to change maxMemory xml tag. It also changes the memory API from 'memory: XXX' to 'memory': {'current': XXX, 'maxmemory': YYY}
maxMemory xml tag is not created by default anymore, when a guest is created. Tag will be created only when the user changes maxmemory manually, and if the value is different (higher) of 'current' memory. So in order to performe memory hotplug, an user has to stop guest, modify maxmemory and then start the guest.
Tests were fixed and new ones were included.
Rodrigo Trujillo (4): Add support to edit max memory in Templates Add function to update 'memory' in objectstore Add support to edit max memory in Guest Changes and add tests to support new memory API (templates/guest)
API.json | 41 ++++---- docs/API.md | 29 ++++-- i18n.py | 12 ++- model/templates.py | 64 +++++++++++- model/vms.py | 245 ++++++++++++++++++++++++-------------------- osinfo.py | 16 ++- root.py | 4 + template.conf | 10 +- tests/test_livemigration.py | 6 +- tests/test_model.py | 22 ++-- tests/test_rest.py | 23 ++++- tests/test_template.py | 38 +++++-- tests/test_vmtemplate.py | 26 +++-- utils.py | 46 +++++++++ vmtemplate.py | 75 ++++++-------- 15 files changed, 428 insertions(+), 229 deletions(-)
participants (2)
-
Aline Manera -
Rodrigo Trujillo