April 2015 - Kimchi-devel - oVirt List Archives

RFC - Guest memory usage
by Daniel Henrique Barboza 27 Oct '15

27 Oct '15

I was looking at the following item in the backlog: Guests Stats: Display memory utilization (use virt-df or virt-top or ...) If I understood it right, the idea here is to show the -inner- memory allocation of the guest. If you have a VM with 4Gb of RAM running an Ubuntu, we want to know how much memory the Ubuntu OS and its processes are using. I've done an investigation and I haven't found any tool to accomplish this. "virt-top", "virsh dommemstat" and the libvirt API retrieves the information of the memory usage of the guest relative to the host. In the example mentioned before, supposing that the host has 64Gb of RAM, all these tools would show that the VM is using 12% of the host RAM. They do not dive in the VM and shows the actual mem usage of the Ubuntu and its processes running there. Haven't found anything useful in other MLs and forums. The common answer is 'run top in a terminal inside the VM', which of course does not suit us. My question is: any thoughts about how we can implement this feature? Because I am starting to think that, in the end, this kind of info is strict to the guest OS and can't be polled from the outside. Thanks!

5 5

[RFC] Split Kimchi into Base Framework and Virtualization layers
by Lucio Correia 05 May '15

05 May '15

Hi, Gustavo (on cc) and me have been analyzing how to split Kimchi Virtualization layer form its Base web framework. We would like to leverage all infrastructure kimchi base framework provides (URI routing, error handling, i18n, authentication, API validation, plugin tabs loading) in other projects, and contribute this back to kimchi-ginger project. The general idea for the web framework is that it provides basic UI with login and tabs menu, and each plugin installed extends the menu with some tabs. We started by making kimchi a plugin, like ginger is. We are way advanced on that and already have most of kimchi's UI running as a plugin over an almost fully running base web framework. Ginger needs minimal changes, mainly on import statements. We can provide changes needed back to ginger too. Currently, we are splitting kimchi specific stuff like this way: (after renaming the dir src/kimchi to src/backend *) * we are using the name "backend" just for disambiguation from virtualization layer (plugins/kimchi). We will need to define a better name for that basic framework. One suggestion is to call it "wok", the base where kimchi is cooked. {src/backend => plugins/kimchi}/API.json | 0 {src/backend/control/vm => plugins/kimchi/control}/Makefile.am | 7 +- {src/backend/control/vm => plugins/kimchi/control}/__init__.py | 2 +- {src/backend => plugins/kimchi}/control/config.py | 0 {src/backend => plugins/kimchi}/control/cpuinfo.py | 0 {src/backend => plugins/kimchi}/control/debugreports.py | 0 {src/backend => plugins/kimchi}/control/groups.py | 0 {src/backend => plugins/kimchi}/control/host.py | 0 {src/backend => plugins/kimchi}/control/interfaces.py | 0 {src/backend => plugins/kimchi}/control/networks.py | 0 {src/backend => plugins/kimchi}/control/peers.py | 0 {src/backend => plugins/kimchi}/control/storagepools.py | 0 {src/backend => plugins/kimchi}/control/storageservers.py | 0 {src/backend => plugins/kimchi}/control/storagevolumes.py | 0 {src/backend => plugins/kimchi}/control/templates.py | 0 {src/backend => plugins/kimchi}/control/users.py | 0 {src/backend => plugins/kimchi}/control/vm/Makefile.am | 0 {src/backend => plugins/kimchi}/control/vm/__init__.py | 0 {src/backend => plugins/kimchi}/control/vm/hostdevs.py | 0 {src/backend => plugins/kimchi}/control/vm/ifaces.py | 0 {src/backend => plugins/kimchi}/control/vm/snapshots.py | 0 {src/backend => plugins/kimchi}/control/vm/storages.py | 0 {src/backend => plugins/kimchi}/control/vms.py | 0 {src/backend => plugins/kimchi}/disks.py | 0 {src/backend => plugins/kimchi}/imageinfo.py | 0 {src/backend => plugins/kimchi}/iscsi.py | 0 {src/backend => plugins/kimchi}/isoinfo.py | 0 {src/backend => plugins/kimchi}/kvmusertests.py | 0 {src/backend => plugins/kimchi}/mockmodel.py | 0 {src/backend/control/vm => plugins/kimchi/model}/Makefile.am | 7 +- {src/backend/control/vm => plugins/kimchi/model}/__init__.py | 8 --- {src/backend => plugins/kimchi}/model/config.py | 0 {src/backend => plugins/kimchi}/model/cpuinfo.py | 0 {src/backend => plugins/kimchi}/model/debugreports.py | 0 {src/backend => plugins/kimchi}/model/diskutils.py | 0 {src/backend => plugins/kimchi}/model/featuretests.py | 0 {src/backend => plugins/kimchi}/model/groups.py | 0 {src/backend => plugins/kimchi}/model/host.py | 0 {src/backend => plugins/kimchi}/model/hostdev.py | 0 {src/backend => plugins/kimchi}/model/interfaces.py | 0 {src/backend => plugins/kimchi}/model/libvirtconnection.py | 0 {src/backend => plugins/kimchi}/model/libvirtstoragepool.py | 0 plugins/kimchi/model/model.py | 93 ++++++++++++++++++++++++++++ {src/backend => plugins/kimchi}/model/networks.py | 0 {src/backend => plugins/kimchi}/model/peers.py | 0 {src/backend => plugins/kimchi}/model/storagepools.py | 0 {src/backend => plugins/kimchi}/model/storageservers.py | 0 {src/backend => plugins/kimchi}/model/storagetargets.py | 0 {src/backend => plugins/kimchi}/model/storagevolumes.py | 0 {src/backend => plugins/kimchi}/model/templates.py | 0 {src/backend => plugins/kimchi}/model/users.py | 0 {src/backend => plugins/kimchi}/model/utils.py | 0 {src/backend => plugins/kimchi}/model/vmhostdevs.py | 0 {src/backend => plugins/kimchi}/model/vmifaces.py | 0 {src/backend => plugins/kimchi}/model/vms.py | 0 {src/backend => plugins/kimchi}/model/vmsnapshots.py | 0 {src/backend => plugins/kimchi}/model/vmstorages.py | 0 {src/backend => plugins/kimchi}/osinfo.py | 0 {src/backend => plugins/kimchi}/scan.py | 0 {config => plugins/kimchi/ui}/Makefile.am | 0 {config/ui => plugins/kimchi/ui/config}/Makefile.am | 0 config/ui/tabs.xml => plugins/kimchi/ui/config/tab-ext.xml | 0 {src/backend => plugins/kimchi}/vmtemplate.py | 0 The remaining generic stuff on the "backend" side, usable or already in use by other plugins, would be: backend |___model | |___model.py | |___plugins.py | |___tasks.py |___control | |___base.py | |___plugins.py | |___tasks.py | |___utils.py |___xmlutils | |___network.py | |___disk.py | |___utils.py | |___graphics.py | |___qemucmdline.py | |___interface.py |___asynctask.py |___auth.py |___basemodel.py |___cachebust.py |___distroloader.py |___exception.py |___i18n.py |___netinfo.py |___network.py |___objectstore.py |___proxy.py |___repositories.py |___rollbackcontext.py |___root.py |___screenshot.py |___server.py |___sslcert.py |___swupdate.py |___template.py |___utils.py |___vnc.py We don't want to make this a too long e-mail and lose the focus on the idea. We have further technical details to discuss in a follow up. Please let us know your thoughts on that, any directions and requirements you have on this, and how could we proceed with that plan. -- Lucio Correia Software Engineer IBM LTC Brazil

2 3

[kimchi-devel][PATCHv4 0/7] Upload storage volume
by lvroyce＠linux.vnet.ibm.com 04 May '15

04 May '15

From: Royce Lv <lvroyce(a)linux.vnet.ibm.com> How to upload: POST /storagepools/<pool-name>/storagevolumes/ {'capacity':size-of-the-volume,'name':name-of-the-volume,'format':'raw'} PUT /storagepools/<pool-name>/storagevolumes/<volume-name> {'chunk':chunk-data,'chunk_size':size-of-chunk,'index':index-of-the-chunk-of-uploaded-file} PATCH 7/7 is a script against real model server, you can use it to verify the logic of upload. Royce Lv (7): Update docs and json schema of storage volume upload Update controller to make update accept formdata params Add lock facility for storage volume upload Update model for storage volume update Fix incomplete record when uploading update test case for storage volume upload A test script against real model server docs/API.md | 7 ++- src/kimchi/API.json | 22 ++++++++ src/kimchi/control/base.py | 6 +-- src/kimchi/i18n.py | 4 ++ src/kimchi/isoinfo.py | 5 +- src/kimchi/model/storagevolumes.py | 61 ++++++++++----------- src/kimchi/model/utils.py | 46 ++++++++++++++++ tests/test_api.py | 108 +++++++++++++++++++++++++++++++++++++ tests/test_model_storagevolume.py | 95 ++++++++++++++++++++++++-------- 9 files changed, 291 insertions(+), 63 deletions(-) create mode 100644 tests/test_api.py -- 2.1.0

3 15

[PATCH 0/5] Support vm name with special characters
by Ramon Medeiros 04 May '15

04 May '15

Ramon Medeiros (5): Create title tag Change the way that get_vm() and vm_list() search for VMs Use get_vm() instead of conn.lookupByName Encode vm_name to generate VNC token Reflect the unicode conversion on token creation src/kimchi/model/vms.py | 50 +++++++++++++++++++++++++++++++++--------- src/kimchi/model/vmstorages.py | 4 ++-- src/kimchi/vmtemplate.py | 4 +++- ui/js/src/kimchi.api.js | 4 ++-- 4 files changed, 47 insertions(+), 15 deletions(-) -- 2.1.0

2 11

[PATCH v2] Supress error messages while checking vm metadata
by Ramon Medeiros 04 May '15

04 May '15

Changes v2: Use parse functions present on model/utils.py The function FeatureTests.disable_libvirt_error_logging only removes the error message that will be displayed on the console. Libvirt also logs errors on messages log. To avoid flood messages log with metadata errors, it will be verified manually if the xml has the tag. Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> --- src/kimchi/model/utils.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/kimchi/model/utils.py b/src/kimchi/model/utils.py index b2739b2..1b114f6 100644 --- a/src/kimchi/model/utils.py +++ b/src/kimchi/model/utils.py @@ -95,6 +95,9 @@ def _kimchi_set_metadata_node(dom, node): def libvirt_get_kimchi_metadata_node(dom, mode="current"): + if _kimchi_get_metadata_node(dom, "metadata") == None: + return + FeatureTests.disable_libvirt_error_logging() try: xml = dom.metadata(libvirt.VIR_DOMAIN_METADATA_ELEMENT, -- 2.1.0

2 1

[PATCH 0/3] Snapshot enhancements
by Crístian Deives 04 May '15

04 May '15

Crístian Deives (3): snapshot: Handle non-existing snapshots in mock lookup snapshot: Change the action 'revert' to a Task snapshot: Allow creating a snapshot of a domain in any state src/kimchi/control/vm/snapshots.py | 2 +- src/kimchi/i18n.py | 1 - src/kimchi/mockmodel.py | 14 ++++++++++++++ src/kimchi/model/vmsnapshots.py | 36 ++++++++++++++++++++++-------------- tests/test_model.py | 15 +++++++++++---- tests/test_rest.py | 13 +++++++++++++ 6 files changed, 61 insertions(+), 20 deletions(-) -- 2.1.0

3 9

[PATCH] Do not resolve hostname to IP in using remote ISOs
by Aline Manera 04 May '15

04 May '15

Due a limitation in old versions of qemu-io (qemu-io was not able to resolve hostnames preveting the ISO streaming support), Kimchi automatically resolved the hostname to IP to workaround this problem. As all the most used Linux distributions (RHEl7.1, Ubuntu 14.10, Fedora 21 and openSUSE 13.2) have a working version of qemu-io, Kimchi does not need to care about it anymore. It also fixes the issue #436 Signed-off-by: Aline Manera <alinefm(a)linux.vnet.ibm.com> Signed-off-by: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com> --- src/kimchi/model/config.py | 2 -- src/kimchi/model/featuretests.py | 26 -------------------------- src/kimchi/model/utils.py | 11 ----------- src/kimchi/model/vms.py | 1 - src/kimchi/model/vmstorages.py | 10 +++------- src/kimchi/vmtemplate.py | 12 ++---------- tests/test_model.py | 11 +---------- 7 files changed, 6 insertions(+), 67 deletions(-) diff --git a/src/kimchi/model/config.py b/src/kimchi/model/config.py index a417c9b..ca0a74b 100644 --- a/src/kimchi/model/config.py +++ b/src/kimchi/model/config.py @@ -51,7 +51,6 @@ class CapabilitiesModel(object): def __init__(self, **kargs): self.conn = kargs['conn'] self.qemu_stream = False - self.qemu_stream_dns = False self.libvirt_stream_protocols = [] self.fc_host_support = False self.metadata_support = False @@ -88,7 +87,6 @@ class CapabilitiesModel(object): kimchi_log.info("*** Running feature tests ***") conn = self.conn.get() self.qemu_stream = FeatureTests.qemu_supports_iso_stream() - self.qemu_stream_dns = FeatureTests.qemu_iso_stream_dns() self.nfs_target_probe = FeatureTests.libvirt_support_nfs_probe(conn) self.fc_host_support = FeatureTests.libvirt_support_fc_host(conn) self.metadata_support = FeatureTests.has_metadata_support(conn) diff --git a/src/kimchi/model/featuretests.py b/src/kimchi/model/featuretests.py index 8cbe31d..9400151 100644 --- a/src/kimchi/model/featuretests.py +++ b/src/kimchi/model/featuretests.py @@ -21,9 +21,7 @@ import cherrypy import libvirt import lxml.etree as ET import platform -import socket import subprocess -import threading from lxml.builder import E @@ -153,30 +151,6 @@ class FeatureTests(object): return len(stderr) == 0 @staticmethod - @servermethod - def qemu_iso_stream_dns(): - host = socket.getfqdn(cherrypy.server.socket_host) - port = cherrypy.server.socket_port - cmd = ["qemu-io", "-r", "http://%s:%d/images/icon-fedora.png" % - (host, port), "-c", "'read -v 0 512'"] - proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - - thread = threading.Thread(target=proc.communicate) - thread.start() - thread.join(5) - - if thread.is_alive(): - proc.kill() - thread.join() - return False - - if proc.returncode != 0: - return False - - return True - - @staticmethod def libvirt_support_fc_host(conn): try: FeatureTests.disable_libvirt_error_logging() diff --git a/src/kimchi/model/utils.py b/src/kimchi/model/utils.py index b2739b2..0c9d519 100644 --- a/src/kimchi/model/utils.py +++ b/src/kimchi/model/utils.py @@ -18,8 +18,6 @@ # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA import libvirt -import socket -import urlparse from lxml import etree, objectify from lxml.builder import E, ElementMaker @@ -41,15 +39,6 @@ def get_vm_name(vm_name, t_name, name_list): raise OperationFailed("KCHUTILS0003E") -def check_remote_disk_path(path, qemu_stream_dns): - hostname = urlparse.urlparse(path).hostname - if hostname is not None and not qemu_stream_dns: - ip = socket.gethostbyname(hostname) - return path.replace(hostname, ip) - - return path - - def get_vm_config_flag(dom, mode="persistent"): # libvirt.VIR_DOMAIN_AFFECT_CURRENT is 0 # VIR_DOMAIN_AFFECT_LIVE is 1, VIR_DOMAIN_AFFECT_CONFIG is 2 diff --git a/src/kimchi/model/vms.py b/src/kimchi/model/vms.py index c8267c1..a6ca27b 100644 --- a/src/kimchi/model/vms.py +++ b/src/kimchi/model/vms.py @@ -125,7 +125,6 @@ class VMsModel(object): stream_protocols = self.caps.libvirt_stream_protocols xml = t.to_vm_xml(name, vm_uuid, libvirt_stream_protocols=stream_protocols, - qemu_stream_dns=self.caps.qemu_stream_dns, graphics=graphics, volumes=vol_list) diff --git a/src/kimchi/model/vmstorages.py b/src/kimchi/model/vmstorages.py index 37aca64..87c6b3d 100644 --- a/src/kimchi/model/vmstorages.py +++ b/src/kimchi/model/vmstorages.py @@ -1,7 +1,7 @@ # # Project Kimchi # -# Copyright IBM, Corp. 2014 +# Copyright IBM, Corp. 2014-2015 # # This library is free software; you can redistribute it and/or # modify it under the terms of the GNU Lesser General Public @@ -26,7 +26,7 @@ from kimchi.exception import OperationFailed from kimchi.model.config import CapabilitiesModel from kimchi.model.vms import DOM_STATE_MAP, VMModel from kimchi.model.storagevolumes import StorageVolumeModel -from kimchi.model.utils import check_remote_disk_path, get_vm_config_flag +from kimchi.model.utils import get_vm_config_flag from kimchi.osinfo import lookup from kimchi.model.diskutils import get_disk_ref_cnt, set_disk_ref_cnt from kimchi.utils import kimchi_log @@ -131,8 +131,6 @@ class VMStoragesModel(object): params['disk'] = vol_info['type'] params.update(self._get_available_bus_address(params['bus'], vm_name)) - params['path'] = check_remote_disk_path(params['path'], - self.caps.qemu_stream_dns) # Add device to VM dev, xml = get_disk_xml(params) @@ -215,9 +213,7 @@ class VMStorageModel(object): if dev_info['type'] != 'cdrom': raise InvalidOperation("KCHVMSTOR0006E") - params['path'] = check_remote_disk_path(params.get('path', ''), - self.caps.qemu_stream_dns) - + params['path'] = params.get('path', '') old_disk_path = dev_info['path'] new_disk_path = params['path'] if new_disk_path != old_disk_path: diff --git a/src/kimchi/vmtemplate.py b/src/kimchi/vmtemplate.py index ef97d83..e047228 100644 --- a/src/kimchi/vmtemplate.py +++ b/src/kimchi/vmtemplate.py @@ -18,7 +18,6 @@ # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA import os -import socket import stat import time import urlparse @@ -128,7 +127,7 @@ class VMTemplate(object): except IsoFormatError: raise InvalidParameter("KCHISO0001E", {'filename': iso}) - def _get_cdrom_xml(self, libvirt_stream_protocols, qemu_stream_dns): + def _get_cdrom_xml(self, libvirt_stream_protocols): if 'cdrom' not in self.info: return '' @@ -139,11 +138,6 @@ class VMTemplate(object): params['index'] = self.info['cdrom_index'] params['path'] = self.info['cdrom'] - hostname = urlparse.urlparse(params['path']).hostname - if hostname is not None and not qemu_stream_dns: - ip = socket.gethostbyname(hostname) - params['path'] = params['path'].replace(hostname, ip) - if self.info.get('iso_stream', False): protocol = urlparse.urlparse(params['path']).scheme if protocol not in libvirt_stream_protocols: @@ -304,10 +298,8 @@ class VMTemplate(object): graphics.update(kwargs.get('graphics', {})) params['graphics'] = get_graphics_xml(graphics) - qemu_stream_dns = kwargs.get('qemu_stream_dns', False) libvirt_stream_protocols = kwargs.get('libvirt_stream_protocols', []) - cdrom_xml = self._get_cdrom_xml(libvirt_stream_protocols, - qemu_stream_dns) + cdrom_xml = self._get_cdrom_xml(libvirt_stream_protocols) if not urlparse.urlparse(self.info.get('cdrom', "")).scheme in \ libvirt_stream_protocols and \ diff --git a/tests/test_model.py b/tests/test_model.py index 210adfd..bd195b5 100644 --- a/tests/test_model.py +++ b/tests/test_model.py @@ -23,9 +23,7 @@ import os import pwd import re import shutil -import socket import time -import urlparse import unittest import uuid @@ -564,14 +562,7 @@ class ModelTests(unittest.TestCase): {'path': valid_remote_iso_path}) cdrom_info = inst.vmstorage_lookup(vm_name, cdrom_dev) cur_cdrom_path = re.sub(":80/", '/', cdrom_info['path']) - - # As Kimchi server is not running during this test case - # CapabilitiesModel.qemu_stream_dns will be always False - # so we need to convert the hostname to IP - output = urlparse.urlparse(valid_remote_iso_path) - hostname = socket.gethostbyname(output.hostname) - url = valid_remote_iso_path.replace(output.hostname, hostname) - self.assertEquals(url, cur_cdrom_path) + self.assertEquals(valid_remote_iso_path, cur_cdrom_path) @unittest.skipUnless(utils.running_as_root(), 'Must be run as root') def test_vm_storage_provisioning(self): -- 2.1.0

2 2

[PATCH 0/7] Make Template defaults configurable
by Aline Manera 30 Apr '15

30 Apr '15

Aline Manera (7): Remove useless variable in osinfo.py Merge common_spec wiht defaults value in osinfo.py Make Template defaults configurable Add libvirt-daemon-config-network package as Kimchi dependency Verify all networks set as Template defaults prior to server start up Create option to auto create ISO pool or not on server start up Verify storage pool set as Template default prior to server starts up contrib/DEBIAN/control.in | 1 + contrib/kimchi.spec.fedora.in | 3 ++ contrib/kimchi.spec.suse.in | 3 ++ docs/README.md | 14 +++--- src/Makefile.am | 2 +- src/kimchi.conf.in | 3 ++ src/kimchi/config.py.in | 1 + src/kimchi/model/model.py | 44 +---------------- src/kimchi/model/networks.py | 61 ++++++++++------------- src/kimchi/model/storagepools.py | 66 +++++++++++++++++++++++++ src/kimchi/model/templates.py | 2 - src/kimchi/osinfo.py | 101 ++++++++++++++++++++++++++++++--------- src/kimchid.in | 3 ++ src/template.conf | 47 ++++++++++++++++++ 14 files changed, 240 insertions(+), 111 deletions(-) create mode 100644 src/template.conf -- 2.1.0

2 15

[PATCH 0/6 v7] Asyn VM creation
by Aline Manera 30 Apr '15

30 Apr '15

V6 -> V7: - Update test cases - Update UI to reuse the clone box for pending VM creation Aline Manera (1): Create VMs Asynchronously: UI Christy Perez (5): Append clone to target_uri for vm clone task Tests for new clone target_uri UI changes for new clone target_uri Create VMs asynchronously: Backend Create VMs Asynchronously: Tests src/kimchi/control/vms.py | 4 +-- src/kimchi/model/vms.py | 33 ++++++++++++++++++----- tests/test_authorization.py | 25 +++++++++++------- tests/test_mockmodel.py | 14 +++++++--- tests/test_model.py | 49 +++++++++++++++++++++++----------- tests/test_rest.py | 60 +++++++++++++++++++++++++++++++----------- tests/test_server.py | 26 ++++++++---------- ui/css/theme-default/list.css | 8 +++--- ui/js/src/kimchi.guest_main.js | 32 ++++++++++++++++++---- ui/pages/guest.html.tmpl | 4 +-- ui/pages/i18n.json.tmpl | 1 + 11 files changed, 176 insertions(+), 80 deletions(-) -- 2.1.0

2 9

kimchi Chef cookbook
by Scott Hain 30 Apr '15

30 Apr '15

Hello, I wanted to share that I've put together a Chef cookbook for running a local dev/test instance of kimchi. I am using it for my own testing and I thought it might be useful to someone else. https://github.com/opscode-cookbooks/kimchi I created for internal testing, but it could easily be modified to set up hardware or other VMs (it may work already for these use cases, but I haven't tested them). Thanks! -- *Scott Hain* — Software Engineer II 360.360.7433 – shain(a)chef.io – *my: *Linkedin <http://www.linkedin.com/> Twitter <http://www.twitter.com/elejia> *CHEF* CHEF.IO <http://chef.io> TM *chef.io* <http://www.chef.io/> Blog <http://www.chef.io/blog/> Facebook <https://www.facebook.com/getchefdotcom> Twitter <https://twitter.com/chef> Youtube <https://www.youtube.com/getchef>

2 1

[PATCH 5/6] Create VMs Asynchronously: Tests
by Christy Perez 28 Apr '15

28 Apr '15

Signed-off-by: Christy Perez <christy(a)linux.vnet.ibm.com> --- tests/test_authorization.py | 23 +++++++++++------- tests/test_mockmodel.py | 12 +++++++--- tests/test_model.py | 46 +++++++++++++++++++++++++---------- tests/test_rest.py | 58 ++++++++++++++++++++++++++++++++++----------- 4 files changed, 100 insertions(+), 39 deletions(-) diff --git a/tests/test_authorization.py b/tests/test_authorization.py index 4fcc496..eae837c 100644 --- a/tests/test_authorization.py +++ b/tests/test_authorization.py @@ -26,7 +26,7 @@ import kimchi.mockmodel from iso_gen import construct_fake_iso from utils import get_free_port, patch_auth, request -from utils import run_server +from utils import run_server, wait_task test_server = None @@ -118,19 +118,24 @@ def test_nonroot_access(self): # Non-root users can only get vms authorized to them model.templates_create({'name': u'test', 'cdrom': fake_iso}) - model.vms_create({'name': u'test-me', 'template': '/templates/test'}) + task_info = model.vms_create({'name': u'test-me', + 'template': '/templates/test'}) + wait_task(model.task_lookup, task_info['id']) + model.vm_update(u'test-me', {'users': [kimchi.mockmodel.fake_user.keys()[0]], 'groups': []}) - model.vms_create({'name': u'test-usera', - 'template': '/templates/test'}) + task_info = model.vms_create({'name': u'test-usera', + 'template': '/templates/test'}) + wait_task(model.task_lookup, task_info['id']) non_root = list(set(model.users_get_list()) - set(['root']))[0] model.vm_update(u'test-usera', {'users': [non_root], 'groups': []}) - model.vms_create({'name': u'test-groupa', - 'template': '/templates/test'}) + task_info = model.vms_create({'name': u'test-groupa', + 'template': '/templates/test'}) + wait_task(model.task_lookup, task_info['id']) a_group = model.groups_get_list()[0] model.vm_update(u'test-groupa', {'groups': [a_group]}) @@ -143,9 +148,9 @@ def test_nonroot_access(self): self.assertEquals(403, resp.status) # Create a vm using mockmodel directly to test Resource access - model.vms_create({'name': 'kimchi-test', - 'template': '/templates/test'}) - + task_info = model.vms_create({'name': 'kimchi-test', + 'template': '/templates/test'}) + wait_task(model.task_lookup, task_info['id']) resp = self.request('/vms/kimchi-test', '{}', 'PUT') self.assertEquals(403, resp.status) resp = self.request('/vms/kimchi-test', '{}', 'DELETE') diff --git a/tests/test_mockmodel.py b/tests/test_mockmodel.py index aa48dd1..a731579 100644 --- a/tests/test_mockmodel.py +++ b/tests/test_mockmodel.py @@ -66,7 +66,9 @@ def test_screenshot_refresh(self): req = json.dumps({'name': 'test', 'cdrom': fake_iso}) request(host, ssl_port, '/templates', req, 'POST') req = json.dumps({'name': 'test-vm', 'template': '/templates/test'}) - request(host, ssl_port, '/vms', req, 'POST') + resp = request(host, ssl_port, '/vms', req, 'POST') + task = json.loads(resp.read()) + wait_task(model.task_lookup, task['id']) # Test screenshot refresh for running vm request(host, ssl_port, '/vms/test-vm/start', '{}', 'POST') @@ -94,7 +96,9 @@ def test_vm_list_sorted(self): def add_vm(name): # Create a VM req = json.dumps({'name': name, 'template': '/templates/test'}) - request(host, ssl_port, '/vms', req, 'POST') + task = json.loads(request(host, ssl_port, '/vms', req, + 'POST').read()) + wait_task(model.task_lookup, task['id']) vms = [u'abc', u'bca', u'cab', u'xba'] for vm in vms: @@ -106,7 +110,9 @@ def add_vm(name): def test_vm_info(self): model.templates_create({'name': u'test', 'cdrom': fake_iso}) - model.vms_create({'name': u'test-vm', 'template': '/templates/test'}) + task = model.vms_create({'name': u'test-vm', + 'template': '/templates/test'}) + wait_task(model.task_lookup, task['id']) vms = model.vms_get_list() self.assertEquals(2, len(vms)) self.assertIn(u'test-vm', vms) diff --git a/tests/test_model.py b/tests/test_model.py index ad0dccd..7e6090e 100644 --- a/tests/test_model.py +++ b/tests/test_model.py @@ -129,8 +129,11 @@ def test_vm_lifecycle(self): rollback.prependDefer(inst.template_delete, 'test') params = {'name': 'kimchi-vm', 'template': '/templates/test'} - inst.vms_create(params) + task = inst.vms_create(params) rollback.prependDefer(inst.vm_delete, 'kimchi-vm') + inst.task_wait(task['id'], 10) + task = inst.task_lookup(task['id']) + self.assertEquals('finished', task['status']) vms = inst.vms_get_list() self.assertTrue('kimchi-vm' in vms) @@ -267,7 +270,8 @@ def test_image_based_template(self): session.store('template', tmpl_name, tmpl_info) params = {'name': 'kimchi-vm', 'template': '/templates/img-tmpl'} - inst.vms_create(params) + task = inst.vms_create(params) + inst.task_wait(task['id']) rollback.prependDefer(inst.vm_delete, 'kimchi-vm') vms = inst.vms_get_list() @@ -286,7 +290,8 @@ def test_vm_graphics(self): inst.templates_create(params) with RollbackContext() as rollback: params = {'name': 'kimchi-vnc', 'template': '/templates/test'} - inst.vms_create(params) + task1 = inst.vms_create(params) + inst.task_wait(task1['id']) rollback.prependDefer(inst.vm_delete, 'kimchi-vnc') info = inst.vm_lookup('kimchi-vnc') @@ -296,7 +301,8 @@ def test_vm_graphics(self): graphics = {'type': 'spice', 'listen': '127.0.0.1'} params = {'name': 'kimchi-spice', 'template': '/templates/test', 'graphics': graphics} - inst.vms_create(params) + task2 = inst.vms_create(params) + inst.task_wait(task2['id']) rollback.prependDefer(inst.vm_delete, 'kimchi-spice') info = inst.vm_lookup('kimchi-spice') @@ -312,6 +318,10 @@ def test_vm_ifaces(self): params = {'name': 'test', 'disks': [], 'cdrom': UBUNTU_ISO} inst.templates_create(params) rollback.prependDefer(inst.template_delete, 'test') + params = {'name': 'kimchi-ifaces', 'template': '/templates/test'} + task = inst.vms_create(params) + inst.task_wait(task['id']) + rollback.prependDefer(inst.vm_delete, 'kimchi-ifaces') # Create a network net_name = 'test-network' @@ -422,7 +432,8 @@ def _attach_disk(expect_bus=modern_disk_bus): inst.templates_create(params) rollback.prependDefer(inst.template_delete, 'test') params = {'name': vm_name, 'template': '/templates/test'} - inst.vms_create(params) + task1 = inst.vms_create(params) + inst.task_wait(task1['id']) rollback.prependDefer(inst.vm_delete, vm_name) prev_count = len(inst.vmstorages_get_list(vm_name)) @@ -465,7 +476,8 @@ def _attach_disk(expect_bus=modern_disk_bus): rollback.prependDefer(inst.template_delete, 'old_distro_template') params = {'name': vm_name, 'template': '/templates/old_distro_template'} - inst.vms_create(params) + task2 = inst.vms_create(params) + inst.task_wait(task2['id']) rollback.prependDefer(inst.vm_delete, vm_name) # Need to check the right disk_bus for old distro @@ -486,7 +498,8 @@ def test_vm_cdrom(self): inst.templates_create(params) rollback.prependDefer(inst.template_delete, 'test') params = {'name': vm_name, 'template': '/templates/test'} - inst.vms_create(params) + task = inst.vms_create(params) + inst.task_wait(task['id']) rollback.prependDefer(inst.vm_delete, vm_name) prev_count = len(inst.vmstorages_get_list(vm_name)) @@ -575,7 +588,8 @@ def test_vm_storage_provisioning(self): rollback.prependDefer(inst.template_delete, 'test') params = {'name': 'test-vm-1', 'template': '/templates/test'} - inst.vms_create(params) + task = inst.vms_create(params) + inst.task_wait(task['id']) rollback.prependDefer(inst.vm_delete, 'test-vm-1') vm_info = inst.vm_lookup(params['name']) @@ -596,10 +610,12 @@ def test_vm_edit(self): with RollbackContext() as rollback: params_1 = {'name': 'kimchi-vm1', 'template': '/templates/test'} params_2 = {'name': 'kimchi-vm2', 'template': '/templates/test'} - inst.vms_create(params_1) + task1 = inst.vms_create(params_1) + inst.task_wait(task1['id']) rollback.prependDefer(utils.rollback_wrapper, inst.vm_delete, 'kimchi-vm1') - inst.vms_create(params_2) + task2 = inst.vms_create(params_2) + inst.task_wait(task2['id']) rollback.prependDefer(utils.rollback_wrapper, inst.vm_delete, 'kimchi-vm2') @@ -778,11 +794,13 @@ def test_delete_running_vm(self): rollback.prependDefer(inst.template_delete, 'test') params = {'name': u'kīмсhī-∨м', 'template': u'/templates/test'} - inst.vms_create(params) + task = inst.vms_create(params) + inst.task_wait(task['id']) rollback.prependDefer(utils.rollback_wrapper, inst.vm_delete, u'kīмсhī-∨м') inst.vm_start(u'kīмсhī-∨м') + self.assertEquals(inst.vm_lookup(u'kīмсhī-∨м')['state'], 'running') rollback.prependDefer(utils.rollback_wrapper, inst.vm_poweroff, u'kīмсhī-∨м') @@ -801,7 +819,8 @@ def test_vm_list_sorted(self): rollback.prependDefer(inst.template_delete, 'test') params = {'name': 'kimchi-vm', 'template': '/templates/test'} - inst.vms_create(params) + task = inst.vms_create(params) + inst.task_wait(task['id']) rollback.prependDefer(inst.vm_delete, 'kimchi-vm') vms = inst.vms_get_list() @@ -873,7 +892,8 @@ def test_use_test_host(self): params = {'name': 'kimchi-vm', 'template': '/templates/test'} - inst.vms_create(params) + task = inst.vms_create(params) + inst.task_wait(task['id']) rollback.prependDefer(inst.vm_delete, 'kimchi-vm') vms = inst.vms_get_list() diff --git a/tests/test_rest.py b/tests/test_rest.py index 65c3db5..8826424 100644 --- a/tests/test_rest.py +++ b/tests/test_rest.py @@ -112,7 +112,9 @@ def test_get_vms(self): req = json.dumps({'name': name, 'template': '/templates/test', 'users': test_users, 'groups': test_groups}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) vms = json.loads(self.request('/vms').read()) self.assertEquals(11, len(vms)) @@ -130,7 +132,9 @@ def test_edit_vm(self): req = json.dumps({'name': 'vm-1', 'template': '/templates/test'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) vm = json.loads(self.request('/vms/vm-1').read()) self.assertEquals('vm-1', vm['name']) @@ -247,7 +251,9 @@ def test_vm_lifecycle(self): # Create a VM req = json.dumps({'name': 'test-vm', 'template': '/templates/test'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) + self.assertEquals(202, resp.status) # Verify the VM vm = json.loads(self.request('/vms/test-vm').read()) @@ -429,7 +435,9 @@ def test_vm_graphics(self): # Create a VM with default args req = json.dumps({'name': 'test-vm', 'template': '/templates/test'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Verify the VM vm = json.loads(self.request('/vms/test-vm').read()) self.assertEquals('127.0.0.1', vm['graphics']['listen']) @@ -443,7 +451,9 @@ def test_vm_graphics(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test', 'graphics': graphics}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Verify the VM vm = json.loads(self.request('/vms/test-vm').read()) self.assertEquals('127.0.0.1', vm['graphics']['listen']) @@ -457,7 +467,9 @@ def test_vm_graphics(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test', 'graphics': graphics}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Verify the VM vm = json.loads(self.request('/vms/test-vm').read()) self.assertEquals('fe00::0', vm['graphics']['listen']) @@ -471,7 +483,9 @@ def test_vm_graphics(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test', 'graphics': graphics}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Verify the VM vm = json.loads(self.request('/vms/test-vm').read()) self.assertEquals('127.0.0.1', vm['graphics']['listen']) @@ -513,7 +527,9 @@ def test_vm_storage_devices(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Delete the VM rollback.prependDefer(self.request, '/vms/test-vm', '{}', 'DELETE') @@ -658,7 +674,9 @@ def test_vm_iface(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Delete the VM rollback.prependDefer(self.request, '/vms/test-vm', '{}', 'DELETE') @@ -738,7 +756,10 @@ def test_vm_customise_storage(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test', 'storagepool': '/storagepools/alt'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) + resp = self.request('/vms/test-vm', {}, 'GET') vm_info = json.loads(resp.read()) # Test template not changed after vm customise its pool @@ -791,7 +812,9 @@ def test_scsi_fc_storage(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test_fc_pool'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Start the VM resp = self.request('/vms/test-vm/start', '{}', 'POST') @@ -816,8 +839,10 @@ def test_unnamed_vms(self): # Create 5 unnamed vms from this template for i in xrange(1, 6): req = json.dumps({'template': '/templates/test'}) - vm = json.loads(self.request('/vms', req, 'POST').read()) - self.assertEquals('test-vm-%i' % i, vm['name']) + task = json.loads(self.request('/vms', req, 'POST').read()) + wait_task(self._task_lookup, task['id']) + resp = self.request('/vms/test-vm-%i' % i, {}, 'GET') + self.assertEquals(resp.status, 200) count = len(json.loads(self.request('/vms').read())) self.assertEquals(6, count) @@ -849,7 +874,10 @@ def test_create_vm_with_img_based_template(self): self.assertEquals(201, resp.status) req = json.dumps({'template': '/templates/test'}) - json.loads(self.request('/vms', req, 'POST').read()) + resp = self.request('/vms', req, 'POST') + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Test storage volume created with backing store of base file resp = json.loads( @@ -939,6 +967,8 @@ def test_screenshot_refresh(self): resp = self.request('/templates', req, 'POST') req = json.dumps({'name': 'test-vm', 'template': '/templates/test'}) resp = self.request('/vms', req, 'POST') + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Test screenshot for shut-off state vm resp = self.request('/vms/test-vm/screenshot') -- 2.1.0

1 0

[PATCH 1/6] Append clone to target_uri for vm clone task
by Christy Perez 28 Apr '15

28 Apr '15

So that we can differentiate between clone and create tasks for VMs. Signed-off-by: Christy Perez <christy(a)linux.vnet.ibm.com> --- src/kimchi/model/vms.py | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/src/kimchi/model/vms.py b/src/kimchi/model/vms.py index a6ca27b..5f8c3da 100644 --- a/src/kimchi/model/vms.py +++ b/src/kimchi/model/vms.py @@ -229,9 +229,8 @@ def clone(self, name): new_name = get_next_clone_name(current_vm_names, name) # create a task with the actual clone function - taskid = add_task(u'/vms/%s' % new_name, self._clone_task, - self.objstore, - {'name': name, 'new_name': new_name}) + taskid = add_task(u'/vms/%s/clone' % new_name, self._clone_task, + self.objstore, {'name': name, 'new_name': new_name}) return self.task.lookup(taskid) -- 2.1.0

1 1

[PATCH v6 0/6] Async VM Creation
by Christy Perez 28 Apr '15

28 Apr '15

v6 Changes: - Rebase v5 Changes: - Rebase - Remove all the unimperative target_uri changes (see below) If a guest has a large disk, and uses a filesystem that requires preallocation, it can take several minutes to create a VM. During that time, kimchi is tied up by the VM creation. This patch changes the VMs Collection to be an AsyncCollection. Another change required for this was to create a more granular way to query vm-related tasks. The original idea was to add another field to the task database, but then Aline suggested just modifying the task_uri. Since the task_uri cretation will be changed in a future patchset, this is now only modified for the conflicting (clone) tasks. Christy Perez (6): Append clone to target_uri for vm clone task Tests for new clone target_uri UI changes for new clone target_uri Create VMs asynchronously: Backend Create VMs Asynchronously: Tests Create VMs Asynchronously: UI src/kimchi/control/vms.py | 4 +-- src/kimchi/model/vms.py | 32 +++++++++++++++++---- tests/test_authorization.py | 23 +++++++++------ tests/test_mockmodel.py | 12 ++++++-- tests/test_model.py | 50 ++++++++++++++++++++++---------- tests/test_model_storagevolume.py | 2 +- tests/test_rest.py | 60 +++++++++++++++++++++++++++++---------- ui/css/theme-default/list.css | 18 ++++++++++++ ui/js/src/kimchi.guest_main.js | 29 +++++++++++++++---- ui/pages/guest.html.tmpl | 3 ++ 10 files changed, 177 insertions(+), 56 deletions(-) -- 2.1.0

1 5

[PATCH 6/6] Create VMs Asynchronously: UI
by Christy Perez 28 Apr '15

28 Apr '15

Signed-off-by: Christy Perez <christy(a)linux.vnet.ibm.com> --- ui/css/theme-default/list.css | 18 ++++++++++++++++++ ui/js/src/kimchi.guest_main.js | 25 ++++++++++++++++++++++--- ui/pages/guest.html.tmpl | 3 +++ 3 files changed, 43 insertions(+), 3 deletions(-) diff --git a/ui/css/theme-default/list.css b/ui/css/theme-default/list.css index 7b32ea6..62d1539 100644 --- a/ui/css/theme-default/list.css +++ b/ui/css/theme-default/list.css @@ -306,3 +306,21 @@ margin-left: 5px; text-shadow: -1px -1px 1px #CCCCCC, 1px 1px 1px #FFFFFF; } + +.guest-create { + margin: 10px; +} + +.guest-create .icon { + background: url('../../images/theme-default/kimchi-loading15x15.gif') no-repeat; + display: inline-block; + width: 20px; + height: 20px; + vertical-align: middle; +} + +.guest-create .text { + color: #666666; + margin-left: 5px; + text-shadow: -1px -1px 1px #CCCCCC, 1px 1px 1px #FFFFFF; +} diff --git a/ui/js/src/kimchi.guest_main.js b/ui/js/src/kimchi.guest_main.js index c712bb4..dbcc162 100644 --- a/ui/js/src/kimchi.guest_main.js +++ b/ui/js/src/kimchi.guest_main.js @@ -201,6 +201,21 @@ kimchi.listVmsAuto = function() { if (kimchi.vmTimeout) { clearTimeout(kimchi.vmTimeout); } + var getCreatingGuests = function(){ + var guests = []; + kimchi.getTasksByFilter('status=running&target_uri='+encodeURIComponent('^/vms/[^/]+$'), function(tasks) { + for(var i=0;i<tasks.length;i++){ + var guestUri = tasks[i].target_uri; + var guestName = guestUri.split('/')[1] + guests.push($.extend({}, kimchi.sampleGuestObject, {name: guestName, isCreating: true})); + if(kimchi.trackingTasks.indexOf(tasks[i].id)==-1) + kimchi.trackTask(tasks[i].id, null, function(err){ + kimchi.message.error(err.message); + }, null); + } + }, null, true); + return guests; + }; var getCloningGuests = function(){ var guests = []; kimchi.getTasksByFilter('status=running&target_uri='+encodeURIComponent('^/vms/.+/clone'), function(tasks) { @@ -219,6 +234,7 @@ kimchi.listVmsAuto = function() { kimchi.listVMs(function(result, textStatus, jqXHR) { if (result && textStatus=="success") { result = getCloningGuests().concat(result); + result = getCreatingGuests().concat(result); if(result.length) { var listHtml = ''; var guestTemplate = kimchi.guestTemplate; @@ -281,7 +297,7 @@ kimchi.createGuestLi = function(vmObject, prevScreenImage, openMenu) { imgLoad.attr('src',load_src); //Link the stopped tile to the start action, the running tile to open the console - if(!vmObject.isCloning){ + if(!(vmObject.isCloning || vmObject.isCreating)){ if (vmRunningBool) { liveTile.off("click", kimchi.vmstart); liveTile.on("click", kimchi.openVmConsole); @@ -329,7 +345,7 @@ kimchi.createGuestLi = function(vmObject, prevScreenImage, openMenu) { } //Setup action event handlers - if(!vmObject.isCloning){ + if(!(vmObject.isCloning || vmObject.isCreating)){ guestActions.find("[name=vm-start]").on({click : kimchi.vmstart}); guestActions.find("[name=vm-poweroff]").on({click : kimchi.vmpoweroff}); if (vmRunningBool) { //If the guest is not running, do not enable reset @@ -362,7 +378,10 @@ kimchi.createGuestLi = function(vmObject, prevScreenImage, openMenu) { }else{ guestActions.find('.btn').attr('disabled', true); - result.find('.guest-clone').removeClass('hide-content'); + if(vmObject.isCloning) + result.find('.guest-clone').removeClass('hide-content'); + else + result.find('.guest-create').removeClass('hide-content'); $('.popover', guestActions.find("div[name=actionmenu]")).remove(); } diff --git a/ui/pages/guest.html.tmpl b/ui/pages/guest.html.tmpl index 17d41ac..aaf41a2 100644 --- a/ui/pages/guest.html.tmpl +++ b/ui/pages/guest.html.tmpl @@ -29,6 +29,9 @@ <div class="guest-clone hide-content"> <span class="icon"></span><span class="text">$_("Cloning")...</span> </div> + <div class="guest-create hide-content"> + <span class="icon"></span><span class="text">$_("Creating")...</span> + </div> </div> <div name="cpu_utilization" class="sortable"> <div class="circleGauge"></div> -- 2.1.0

1 0

[PATCH 4/6] Create VMs asynchronously: Backend
by Christy Perez 28 Apr '15

28 Apr '15

Signed-off-by: Christy Perez <christy(a)linux.vnet.ibm.com> --- src/kimchi/control/vms.py | 4 ++-- src/kimchi/model/vms.py | 27 ++++++++++++++++++++++++--- 2 files changed, 26 insertions(+), 5 deletions(-) diff --git a/src/kimchi/control/vms.py b/src/kimchi/control/vms.py index 6352a26..a40b56e 100644 --- a/src/kimchi/control/vms.py +++ b/src/kimchi/control/vms.py @@ -17,13 +17,13 @@ # License along with this library; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA -from kimchi.control.base import Collection, Resource +from kimchi.control.base import AsyncCollection, Resource from kimchi.control.utils import internal_redirect, UrlSubNode from kimchi.control.vm import sub_nodes @UrlSubNode('vms', True) -class VMs(Collection): +class VMs(AsyncCollection): def __init__(self, model): super(VMs, self).__init__(model) self.resource = VM diff --git a/src/kimchi/model/vms.py b/src/kimchi/model/vms.py index 5f8c3da..9786063 100644 --- a/src/kimchi/model/vms.py +++ b/src/kimchi/model/vms.py @@ -79,9 +79,9 @@ def __init__(self, **kargs): self.conn = kargs['conn'] self.objstore = kargs['objstore'] self.caps = CapabilitiesModel(**kargs) + self.task = TaskModel(**kargs) def create(self, params): - conn = self.conn.get() t_name = template_name_from_uri(params['template']) vm_uuid = str(uuid.uuid4()) vm_list = self.get_list() @@ -102,7 +102,26 @@ def create(self, params): raise InvalidOperation("KCHVM0005E") t.validate() + taskid = add_task(u'/vms/%s' % name, + self._create_task, self.objstore, + {'vm_uuid': vm_uuid, 'template': t, 'name': name}) + + return self.task.lookup(taskid) + + def _create_task(self, cb, params): + """ + params: A dict with the following values: + - vm_uuid: The UUID of the VM being created + - template: The template being used to create the VM + - name: The name for the new VM + """ + + vm_uuid = params['vm_uuid'] + t = params['template'] + name = params['name'] + conn = self.conn.get() + cb('Storing VM icon') # Store the icon for displaying later icon = t.info.get('icon') if icon: @@ -117,6 +136,7 @@ def create(self, params): # If storagepool is SCSI, volumes will be LUNs and must be passed by # the user from UI or manually. + cb('Provisioning storage for new VM') vol_list = [] if t._get_storage_type() not in ["iscsi", "scsi"]: vol_list = t.fork_vm_storage(vm_uuid) @@ -128,6 +148,7 @@ def create(self, params): graphics=graphics, volumes=vol_list) + cb('Defining new VM') try: conn.defineXML(xml.encode('utf-8')) except libvirt.libvirtError as e: @@ -138,10 +159,10 @@ def create(self, params): raise OperationFailed("KCHVM0007E", {'name': name, 'err': e.get_error_message()}) + cb('Updating VM metadata') VMModel.vm_update_os_metadata(VMModel.get_vm(name, self.conn), t.info, self.caps.metadata_support) - - return name + cb('OK', True) def get_list(self): return VMsModel.get_vms(self.conn) -- 2.1.0

1 0

[PATCH 3/6] UI changes for new clone target_uri
by Christy Perez 28 Apr '15

28 Apr '15

Signed-off-by: Christy Perez <christy(a)linux.vnet.ibm.com> --- ui/js/src/kimchi.guest_main.js | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ui/js/src/kimchi.guest_main.js b/ui/js/src/kimchi.guest_main.js index b66177c..c712bb4 100644 --- a/ui/js/src/kimchi.guest_main.js +++ b/ui/js/src/kimchi.guest_main.js @@ -203,10 +203,10 @@ kimchi.listVmsAuto = function() { } var getCloningGuests = function(){ var guests = []; - kimchi.getTasksByFilter('status=running&target_uri='+encodeURIComponent('^/vms/*'), function(tasks) { + kimchi.getTasksByFilter('status=running&target_uri='+encodeURIComponent('^/vms/.+/clone'), function(tasks) { for(var i=0;i<tasks.length;i++){ var guestUri = tasks[i].target_uri; - var guestName = guestUri.substring(guestUri.lastIndexOf('/')+1, guestUri.length); + var guestName = guestUri.split('/')[2] guests.push($.extend({}, kimchi.sampleGuestObject, {name: guestName, isCloning: true})); if(kimchi.trackingTasks.indexOf(tasks[i].id)==-1) kimchi.trackTask(tasks[i].id, null, function(err){ -- 2.1.0

1 0

[PATCH v5 0/6] Async VM Creation
by Christy Perez 28 Apr '15

28 Apr '15

v5 Changes: - Rebase - Remove all the unimperative target_uri changes (see below) If a guest has a large disk, and uses a filesystem that requires preallocation, it can take several minutes to create a VM. During that time, kimchi is tied up by the VM creation. This patch changes the VMs Collection to be an AsyncCollection. Another change required for this was to create a more granular way to query vm-related tasks. The original idea was to add another field to the task database, but then Aline suggested just modifying the task_uri. Since the task_uri cretation will be changed in a future patchset, this is now only modified for the conflicting (clone) tasks. Christy Perez (6): Append clone to target_uri for vm clone task Tests for new clone target_uri UI changes for new clone target_uri Create VMs asynchronously: Backend Create VMs Asynchronously: Tests Create VMs Asynchronously: UI src/kimchi/control/vms.py | 4 +-- src/kimchi/model/vms.py | 32 +++++++++++++++++---- tests/test_authorization.py | 23 +++++++++------ tests/test_mockmodel.py | 12 ++++++-- tests/test_model.py | 49 +++++++++++++++++++++----------- tests/test_model_storagevolume.py | 2 +- tests/test_rest.py | 60 +++++++++++++++++++++++++++++---------- ui/css/theme-default/list.css | 18 ++++++++++++ ui/js/src/kimchi.guest_main.js | 29 +++++++++++++++---- ui/pages/guest.html.tmpl | 3 ++ 10 files changed, 175 insertions(+), 57 deletions(-) -- 2.1.0

2 9

[PATCH] Improve code to retrieve the number of host CPUs
by Jose Ricardo Ziviani 28 Apr '15

28 Apr '15

- Retrieving the number of cpus using psinfo is unstable, there are three (at least) different ways to do it that changes from version to version. This code improves the way such functions are called to avoid any surprise in future. Signed-off-by: Jose Ricardo Ziviani <joserz(a)linux.vnet.ibm.com> --- src/kimchi/model/host.py | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/src/kimchi/model/host.py b/src/kimchi/model/host.py index 293c4ce..b2fa379 100644 --- a/src/kimchi/model/host.py +++ b/src/kimchi/model/host.py @@ -102,15 +102,20 @@ class HostModel(object): return res def lookup(self, *name): - cpus = 0 + cpus = psutil.NUM_CPUS - # Only newer psutil versions have a portable method to get - # the number of cpus - try: + # psutil is unstable on how to get the number of + # cpus, different versions call it differently + if hasattr(psutil, 'cpu_count'): cpus = psutil.cpu_count() - except AttributeError: - cpus = psutil._psplatform.get_num_cpus() + elif hasattr(psutil, '_psplatform'): + for method_name in ['_get_num_cpus', 'get_num_cpus']: + + method = getattr(psutil._psplatform, method_name, None) + if method is not None: + cpus = method() + break self.host_info['cpus'] = cpus self.host_info['memory'] = psutil.phymem_usage().total -- 1.9.1

2 2

[PATCH] Move kimchi nginx config file to nginx default directory.
by Jose Ricardo Ziviani 28 Apr '15

28 Apr '15

- Move Kimchi nginx config file to /etc/nginx/conf.d. - Rename nginx_kimchi.conf to kimchi.conf. --- .gitignore | 2 +- configure.ac | 1 + contrib/kimchi.spec.fedora.in | 6 ++-- contrib/kimchi.spec.suse.in | 7 ++-- src/Makefile.am | 5 ++- src/kimchi/config.py.in | 2 ++ src/kimchi/proxy.py | 12 +++---- src/nginx.conf.in | 76 ------------------------------------------- src/nginx/Makefile.am | 23 +++++++++++++ src/nginx/kimchi.conf.in | 76 +++++++++++++++++++++++++++++++++++++++++++ 10 files changed, 119 insertions(+), 91 deletions(-) delete mode 100644 src/nginx.conf.in create mode 100644 src/nginx/Makefile.am create mode 100644 src/nginx/kimchi.conf.in diff --git a/.gitignore b/.gitignore index 4abfc0a..a318bd9 100644 --- a/.gitignore +++ b/.gitignore @@ -28,7 +28,7 @@ stamp-po kimchi-*.tar.gz src/kimchid src/kimchi.conf -src/nginx_kimchi.conf +src/nginx/kimchi.conf src/kimchi/config.py tests/run_tests.sh tests/test_config.py diff --git a/configure.ac b/configure.ac index 1b476c3..5da3240 100644 --- a/configure.ac +++ b/configure.ac @@ -80,6 +80,7 @@ AC_CONFIG_FILES([ docs/Makefile src/Makefile src/distros.d/Makefile + src/nginx/Makefile src/kimchi/Makefile src/kimchi/control/Makefile src/kimchi/control/vm/Makefile diff --git a/contrib/kimchi.spec.fedora.in b/contrib/kimchi.spec.fedora.in index 750dada..a721a7f 100644 --- a/contrib/kimchi.spec.fedora.in +++ b/contrib/kimchi.spec.fedora.in @@ -94,7 +94,7 @@ touch %{buildroot}/%{_localstatedir}/log/kimchi/kimchi-error.log # create /etc/kimchi structure mkdir -p %{buildroot}/%{_sysconfdir}/kimchi/ -touch %{buildroot}/%{_sysconfdir}/kimchi/nginx_kimchi.conf +touch %{buildroot}/%{_sysconfdir}/nginx/conf.d/kimchi.conf # Install the systemd scripts install -Dm 0644 contrib/kimchid.service.fedora %{buildroot}%{_unitdir}/kimchid.service @@ -158,14 +158,14 @@ rm -rf $RPM_BUILD_ROOT %{_datadir}/kimchi/config/ui/*.xml %{_datadir}/kimchi/ui/ %{_datadir}/kimchi +%{_sysconfdir}/nginx/conf.d/kimchi.conf.in +%{_sysconfdir}/nginx/conf.d/kimchi.conf %{_sysconfdir}/kimchi/kimchi.conf -%{_sysconfdir}/kimchi/nginx.conf.in %{_sysconfdir}/kimchi/distros.d/debian.json %{_sysconfdir}/kimchi/distros.d/fedora.json %{_sysconfdir}/kimchi/distros.d/opensuse.json %{_sysconfdir}/kimchi/distros.d/ubuntu.json %{_sysconfdir}/kimchi/distros.d/gentoo.json -%{_sysconfdir}/kimchi/nginx_kimchi.conf %{_sysconfdir}/kimchi/ %{_sharedstatedir}/kimchi/debugreports/ %{_sharedstatedir}/kimchi/screenshots/ diff --git a/contrib/kimchi.spec.suse.in b/contrib/kimchi.spec.suse.in index 7e4172d..5673ced 100644 --- a/contrib/kimchi.spec.suse.in +++ b/contrib/kimchi.spec.suse.in @@ -63,7 +63,9 @@ touch %{buildroot}/%{_localstatedir}/log/kimchi/kimchi-error.log # create /etc/kimchi structure mkdir -p %{buildroot}/%{_sysconfdir}/kimchi/ -touch %{buildroot}/%{_sysconfdir}/kimchi/nginx_kimchi.conf + +# create nginx configuration +touch %{buildroot}/%{_sysconfdir}/nginx/conf.d/kimchi.conf # Install the SysV init scripts install -Dm 0755 contrib/kimchid.sysvinit %{buildroot}%{_initrddir}/kimchid @@ -97,13 +99,14 @@ rm -rf $RPM_BUILD_ROOT %{_datadir}/kimchi/config/ui/*.xml %{_datadir}/kimchi/ui/ %{_datadir}/kimchi +%{_sysconfdir}/nginx/conf.d/kimchi.conf.in +%{_sysconfdir}/nginx/conf.d/kimchi.conf %{_sysconfdir}/kimchi/kimchi.conf %{_sysconfdir}/kimchi/distros.d/debian.json %{_sysconfdir}/kimchi/distros.d/fedora.json %{_sysconfdir}/kimchi/distros.d/opensuse.json %{_sysconfdir}/kimchi/distros.d/ubuntu.json %{_sysconfdir}/kimchi/distros.d/gentoo.json -%{_sysconfdir}/kimchi/nginx_kimchi.conf %{_sysconfdir}/kimchi %{_initrddir}/kimchid %{_sysconfdir}/kimchi/ diff --git a/src/Makefile.am b/src/Makefile.am index dfeb24e..edc163e 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -17,18 +17,17 @@ # License along with this library; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA -SUBDIRS = kimchi distros.d +SUBDIRS = kimchi distros.d nginx EXTRA_DIST = kimchid.in \ kimchi.conf.in \ - nginx.conf.in \ firewalld.xml \ $(NULL) bin_SCRIPTS = kimchid confdir = $(sysconfdir)/kimchi -dist_conf_DATA = kimchi.conf nginx.conf.in +dist_conf_DATA = kimchi.conf BUILT_SOURCES = kimchi.conf diff --git a/src/kimchi/config.py.in b/src/kimchi/config.py.in index f2e1cac..f15d228 100644 --- a/src/kimchi/config.py.in +++ b/src/kimchi/config.py.in @@ -112,6 +112,7 @@ class Paths(object): self.novnc_dir = '/usr/share/novnc' if self.installed: + self.nginx_conf_dir = '@sysconfdir@/nginx/conf.d' self.state_dir = '@localstatedir@/lib/kimchi' self.log_dir = '@localstatedir@/log/kimchi' self.conf_dir = '@sysconfdir@/kimchi' @@ -120,6 +121,7 @@ class Paths(object): self.mo_dir = '@prefix@/share/locale' self.spice_css_file = os.path.join(self.spice_dir, 'spice.css') else: + self.nginx_conf_dir = self.add_prefix('src/nginx') self.state_dir = self.add_prefix('data') self.log_dir = self.add_prefix('log') self.conf_dir = self.add_prefix('src') diff --git a/src/kimchi/proxy.py b/src/kimchi/proxy.py index fafa5bc..1b70d5e 100644 --- a/src/kimchi/proxy.py +++ b/src/kimchi/proxy.py @@ -37,8 +37,7 @@ def _create_proxy_config(options): To allow flexibility in which port kimchi runs, we need the same flexibility with the nginx proxy. This method creates the config file dynamically by using 'nginx.conf.in' as a template, creating - the file 'nginx_kimchi.config' which will be used to launch the - proxy. + the file 'kimchi.conf' which will be used to launch the proxy. Arguments: options - OptionParser object with Kimchi config options @@ -53,6 +52,7 @@ def _create_proxy_config(options): user_proxy = 'www-data' config_dir = paths.conf_dir + nginx_config_dir = paths.nginx_conf_dir cert = options.ssl_cert key = options.ssl_key @@ -70,7 +70,7 @@ def _create_proxy_config(options): # Read template file and create a new config file # with the specified parameters. - with open(os.path.join(config_dir, "nginx.conf.in")) as template: + with open(os.path.join(nginx_config_dir, "kimchi.conf.in")) as template: data = template.read() data = Template(data) data = data.safe_substitute(user=user_proxy, @@ -81,7 +81,7 @@ def _create_proxy_config(options): max_body_size=eval(options.max_body_size)) # Write file to be used for nginx. - config_file = open(os.path.join(config_dir, "nginx_kimchi.conf"), "w") + config_file = open(os.path.join(nginx_config_dir, "kimchi.conf"), "w") config_file.write(data) config_file.close() @@ -89,8 +89,8 @@ def _create_proxy_config(options): def start_proxy(options): """Start nginx reverse proxy.""" _create_proxy_config(options) - config_dir = paths.conf_dir - config_file = "%s/nginx_kimchi.conf" % config_dir + nginx_config_dir = paths.nginx_conf_dir + config_file = "%s/kimchi.conf" % nginx_config_dir cmd = ['nginx', '-c', config_file] subprocess.call(cmd) diff --git a/src/nginx.conf.in b/src/nginx.conf.in deleted file mode 100644 index e308152..0000000 --- a/src/nginx.conf.in +++ /dev/null @@ -1,76 +0,0 @@ -# Project Kimchi -# -# Copyright IBM, Corp. 2014 -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library; if not, write to the Free Software -# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA -# 02110-1301 USA - -# This is a template file to be used to generate a nginx -# proxy config file at kimchid script. - -user ${user}; -worker_processes 1; - -error_log /var/log/nginx/error.log; - -events { - worker_connections 1024; -} - -http { - - log_format main '$remote_addr - $remote_user [$time_local] "$request" ' - '$status $body_bytes_sent "$http_referer" ' - '"$http_user_agent" "$http_x_forwarded_for"'; - - access_log /var/log/nginx/access.log main; - sendfile on; - - client_max_body_size ${max_body_size}k; - - # Timeout set to 10 minutes to avoid the 504 Gateway Timeout - # when Kimchi is processing a request. - proxy_connect_timeout 600; - proxy_send_timeout 600; - proxy_read_timeout 600; - send_timeout 600; - - server { - listen ${proxy_ssl_port} ssl; - - ssl_certificate ${cert_pem}; - ssl_certificate_key ${cert_key}; - ssl_protocols TLSv1 TLSv1.1 TLSv1.2; - ssl_ciphers ECDH@STRENGTH:DH@STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNULL; - - add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;"; - add_header X-Frame-Options DENY; - add_header X-Content-Type-Options nosniff; - add_header X-XSS-Protection "1; mode=block"; - - location / { - proxy_pass http://127.0.0.1:${kimchid_port}; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_redirect http://127.0.0.1:${kimchid_port}/ https://$host:${proxy_ssl_port}/; - } - } - - server { - listen ${proxy_port}; - rewrite ^/(.*)$ https://$host:${proxy_ssl_port}/$1 redirect; - } -} diff --git a/src/nginx/Makefile.am b/src/nginx/Makefile.am new file mode 100644 index 0000000..b240508 --- /dev/null +++ b/src/nginx/Makefile.am @@ -0,0 +1,23 @@ +# +# Kimchi +# +# Copyright IBM Corp, 2013 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +EXTRA_DIST = kimchi.conf.in + +confdir = $(sysconfdir)/nginx/conf.d +dist_conf_DATA = kimchi.conf.in diff --git a/src/nginx/kimchi.conf.in b/src/nginx/kimchi.conf.in new file mode 100644 index 0000000..e308152 --- /dev/null +++ b/src/nginx/kimchi.conf.in @@ -0,0 +1,76 @@ +# Project Kimchi +# +# Copyright IBM, Corp. 2014 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301 USA + +# This is a template file to be used to generate a nginx +# proxy config file at kimchid script. + +user ${user}; +worker_processes 1; + +error_log /var/log/nginx/error.log; + +events { + worker_connections 1024; +} + +http { + + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; + + access_log /var/log/nginx/access.log main; + sendfile on; + + client_max_body_size ${max_body_size}k; + + # Timeout set to 10 minutes to avoid the 504 Gateway Timeout + # when Kimchi is processing a request. + proxy_connect_timeout 600; + proxy_send_timeout 600; + proxy_read_timeout 600; + send_timeout 600; + + server { + listen ${proxy_ssl_port} ssl; + + ssl_certificate ${cert_pem}; + ssl_certificate_key ${cert_key}; + ssl_protocols TLSv1 TLSv1.1 TLSv1.2; + ssl_ciphers ECDH@STRENGTH:DH@STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNULL; + + add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;"; + add_header X-Frame-Options DENY; + add_header X-Content-Type-Options nosniff; + add_header X-XSS-Protection "1; mode=block"; + + location / { + proxy_pass http://127.0.0.1:${kimchid_port}; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_redirect http://127.0.0.1:${kimchid_port}/ https://$host:${proxy_ssl_port}/; + } + } + + server { + listen ${proxy_port}; + rewrite ^/(.*)$ https://$host:${proxy_ssl_port}/$1 redirect; + } +} -- 1.9.1

2 2

Re: [Kimchi-devel] [PATCH 3/5] Use get_vm() instead of conn.lookupByName
by Ramon Medeiros 28 Apr '15

28 Apr '15

Please ignore these patches On 04/28/2015 10:16 AM, Ramon Medeiros wrote: > Remove duplicated code of conn.lookupByName. > > Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> > --- > src/kimchi/model/vms.py | 2 +- > src/kimchi/model/vmstorages.py | 4 ++-- > 2 files changed, 3 insertions(+), 3 deletions(-) > > diff --git a/src/kimchi/model/vms.py b/src/kimchi/model/vms.py > index 6792447..87d63c1 100644 > --- a/src/kimchi/model/vms.py > +++ b/src/kimchi/model/vms.py > @@ -266,7 +266,7 @@ class VMModel(object): > # fetch base XML > cb('reading source VM XML') > try: > - vir_dom = vir_conn.lookupByName(name) > + vir_dom = self.get_vm(name) > flags = libvirt.VIR_DOMAIN_XML_SECURE > xml = vir_dom.XMLDesc(flags).decode('utf-8') > except libvirt.libvirtError, e: > diff --git a/src/kimchi/model/vmstorages.py b/src/kimchi/model/vmstorages.py > index 37aca64..8dabae7 100644 > --- a/src/kimchi/model/vmstorages.py > +++ b/src/kimchi/model/vmstorages.py > @@ -138,7 +138,7 @@ class VMStoragesModel(object): > dev, xml = get_disk_xml(params) > try: > conn = self.conn.get() > - dom = conn.lookupByName(vm_name) > + dom = VMModel.get_vm(vm_name, conn) > dom.attachDeviceFlags(xml, get_vm_config_flag(dom, 'all')) > except Exception as e: > raise OperationFailed("KCHVMSTOR0008E", {'error': e.message}) > @@ -173,7 +173,7 @@ class VMStorageModel(object): > > try: > bus_type = self.lookup(vm_name, dev_name)['bus'] > - dom = conn.lookupByName(vm_name) > + dom = VMModel.get_vm(vm_name, conn) > except NotFoundError: > raise > -- Ramon Nunes Medeiros Kimchi Developer Linux Technology Center Brazil IBM Systems & Technology Group Phone : +55 19 2132 7878 ramonn(a)br.ibm.com

1 0

[PATCH] Fix disk format lock and checking during template creation
by Rodrigo Trujillo 27 Apr '15

27 Apr '15

Users are able to pass the disk format (qcow, raw, etc) in disk Template information. However, Kimchi was ignoring this information and always creating QCOW2 disk images (if the storagepool is not 'LOGICAL') when it creates a VM based on a given Template. This patch fixes this problem and also moves the checking and disk format auto assignment to Template creation phase, in VMTemplate class init. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/kimchi/i18n.py | 1 + src/kimchi/vmtemplate.py | 15 +++++++++++---- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/src/kimchi/i18n.py b/src/kimchi/i18n.py index 18e84bc..0651032 100644 --- a/src/kimchi/i18n.py +++ b/src/kimchi/i18n.py @@ -154,6 +154,7 @@ messages = { "KCHTMPL0025E": _("When specifying CPU topology, VCPUs must be a product of sockets, cores, and threads."), "KCHTMPL0026E": _("When specifying CPU topology, each element must be an integer greater than zero."), "KCHTMPL0027E": _("Invalid disk image format. Valid formats: bochs, cloop, cow, dmg, qcow, qcow2, qed, raw, vmdk, vpc."), + "KCHTMPL0028E": _("Invalid disk image format. Logical, SCSI and iSCSI storage pools must use disk image format 'raw'."), "KCHPOOL0001E": _("Storage pool %(name)s already exists"), "KCHPOOL0002E": _("Storage pool %(name)s does not exist"), diff --git a/src/kimchi/vmtemplate.py b/src/kimchi/vmtemplate.py index ef97d83..84e877b 100644 --- a/src/kimchi/vmtemplate.py +++ b/src/kimchi/vmtemplate.py @@ -75,8 +75,16 @@ class VMTemplate(object): args['graphics'] = graphics self.info.update(args) - # Assign right disk format to logical and [i]scsi storagepools + # Check disk image format passed and pool type if self._get_storage_type() in ['logical', 'iscsi', 'scsi']: + paramdisks = args.get('disks') + if paramdisks is not None: + for disk in paramdisks: + fmt = disk.get('format') + if fmt is not None and fmt != 'raw': + raise InvalidParameter('KCHTMPL0028E') + # Finally add disk image format when not passed or + # using default for i, disk in enumerate(self.info['disks']): self.info['disks'][i]['format'] = 'raw' @@ -198,7 +206,6 @@ class VMTemplate(object): def to_volume_list(self, vm_uuid): storage_path = self._get_storage_path() - fmt = 'raw' if self._get_storage_type() in ['logical'] else 'qcow2' ret = [] for i, d in enumerate(self.info['disks']): index = d.get('index', i) @@ -206,11 +213,11 @@ class VMTemplate(object): info = {'name': volume, 'capacity': d['size'], - 'format': fmt, + 'format': d['format'], 'path': '%s/%s' % (storage_path, volume)} if 'logical' == self._get_storage_type() or \ - fmt not in ['qcow2', 'raw']: + info['format'] not in ['qcow2', 'raw']: info['allocation'] = info['capacity'] else: info['allocation'] = 0 -- 2.1.0

2 3

[PATCH V2] UI-Template Edit: Enable user to change disk format
by Rodrigo Trujillo 27 Apr '15

27 Apr '15

This patch adds a new field (Disk Format) in Storage tab in Template edit window. Users will then be allowed to select any disk format supported by Libvirt and Kimchi backend. The default disk format is qcow2 (first option), so, for compatibility, if the template register in objecstore does not contain the disk format information, the new field is going to set 'qcow2' automatically. For iscsi storagepools, the new field is going to be disabled and the format is going to be set automatically as 'raw', just like the backend behaves currently, avoiding errors. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- ui/css/theme-default/template-edit.css | 8 ++++-- ui/js/src/kimchi.template_edit_main.js | 52 ++++++++++++++++++++++++++++++---- ui/pages/template-edit.html.tmpl | 16 +++++++++++ 3 files changed, 68 insertions(+), 8 deletions(-) diff --git a/ui/css/theme-default/template-edit.css b/ui/css/theme-default/template-edit.css index 7abee7c..ed9f02c 100644 --- a/ui/css/theme-default/template-edit.css +++ b/ui/css/theme-default/template-edit.css @@ -95,7 +95,7 @@ #edit-template-tabs .template-storage-cell{ display: inline-block; - width: 230px; + width: 180px; } #edit-template-tabs .template-storage-cell label { @@ -114,7 +114,11 @@ } #form-template-storage .template-tab-body .template-storage-name { - width: 220px; + width: 170px; +} + +#form-template-storage .template-tab-body .template-storage-disk-format { + width: 160px; } #edit-template-tabs .template-tab-body input[readonly] { diff --git a/ui/js/src/kimchi.template_edit_main.js b/ui/js/src/kimchi.template_edit_main.js index 85f76cd..6029a95 100644 --- a/ui/js/src/kimchi.template_edit_main.js +++ b/ui/js/src/kimchi.template_edit_main.js @@ -67,7 +67,7 @@ kimchi.template_edit_main = function() { $('.template-tab-body', '#form-template-storage').append(nodeStorage); var storageOptions = ''; var scsiOptions = ''; - $('select', '#form-template-storage').find('option').remove(); + $('select:first', '#form-template-storage').find('option').remove(); $.each(result, function(index, storageEntities) { if((storageEntities.state === 'active') && (storageEntities.type != 'kimchi-iso')) { if(storageEntities.type === 'iscsi' || storageEntities.type === 'scsi') { @@ -77,7 +77,7 @@ kimchi.template_edit_main = function() { var isSlected = tmpPath === thisName ? ' selected' : ''; scsiOptions += '<option' + isSlected + '>' + tmpPath + '</option>'; }); - $('select', '#form-template-storage').append(scsiOptions); + $('select:first', '#form-template-storage').append(scsiOptions); }, function() {}); } else { var isSlected = storageEntities.name === thisName ? ' selected' : ''; @@ -85,8 +85,20 @@ kimchi.template_edit_main = function() { } } }); - $('select', '#form-template-storage').append(storageOptions); - $('select', '#form-template-storage').change(function() { + $('select:first', '#form-template-storage').append(storageOptions); + + // Set disk format + $('select:last option', '#form-template-storage').each(function() { + if ($(this).text() == storageData.storageDiskFormat) { + $(this).prop('selected', true); + } + }); + + $('select:last', '#form-template-storage').change(function() { + $('.template-storage-disk-format').val($(this).val()); + }); + + $('select:first', '#form-template-storage').change(function() { var selectedItem = $(this).parent().parent(); var tempStorageNameFull = $(this).val(); var tempName = tempStorageNameFull.split('/'); @@ -99,9 +111,25 @@ kimchi.template_edit_main = function() { kimchi.getStoragePoolVolume(tempStorageName, tempName[tempName.length-1], function(info) { volSize = info.capacity / Math.pow(1024, 3); $('.template-storage-disk', selectedItem).attr('readonly', true).val(volSize); + $('select:last option', selectedItem).each(function() { + this.selected = (this.text == 'raw'); + }); + $('select:last', selectedItem).prop('disabled', true).change(); }); + } else if (tempType === 'logical') { + $('.template-storage-disk', selectedItem).attr('readonly', false); + $('select:last option', selectedItem).each(function() { + this.selected = (this.text == 'raw'); + }); + $('select:last', selectedItem).prop('disabled', true).change(); } else { $('.template-storage-disk', selectedItem).attr('readonly', false); + if ($('select:last', selectedItem).prop('disabled') == true) { + $('select:last option', selectedItem).each(function() { + this.selected = (this.text == 'qcow2'); + }); + $('select:last', selectedItem).prop('disabled', false).change(); + } } }); }); @@ -120,7 +148,8 @@ kimchi.template_edit_main = function() { editMode : 'hide', storageName : defaultPool, storageType : defaultType, - storageDisk : diskEntities.size + storageDisk : diskEntities.size, + storageDiskFormat : diskEntities.format ? diskEntities.format : 'qcow2' } if (diskEntities.volume) { @@ -131,7 +160,17 @@ kimchi.template_edit_main = function() { nodeData.storageDisk = volSize; addStorageItem(nodeData); $('.template-storage-disk').attr('readonly', true); + $('select:last option', '#form-template-storage').each(function() { + this.selected = (this.text == 'raw'); + }); + $('select:last', '#form-template-storage').prop('disabled', true).change(); + }); + } else if (defaultType === 'logical') { + addStorageItem(storageNodeData); + $('select:last option', '#form-template-storage').each(function() { + this.selected = (this.text == 'raw'); }); + $('select:last', '#form-template-storage').prop('disabled', true).change(); } else { addStorageItem(storageNodeData); } @@ -271,7 +310,8 @@ kimchi.template_edit_main = function() { origDisks[0]['volume'] && delete origDisks[0]['volume']; origDisks[0].size = Number($('.template-storage-disk', tmpItem).val()); } - data[field] = origDisks; + origDisks[0].format = $('.template-storage-disk-format', tmpItem).val(); + data[field] = origDisks; } else if (field == 'graphics') { var type = $('#form-template-general [name="' + field + '"]').val(); diff --git a/ui/pages/template-edit.html.tmpl b/ui/pages/template-edit.html.tmpl index c7832c9..e64a30f 100644 --- a/ui/pages/template-edit.html.tmpl +++ b/ui/pages/template-edit.html.tmpl @@ -104,6 +104,7 @@ <span class="template-storage-cell">$_("Storage Pool")</span> <span class="template-storage-cell">$_("Type")</span> <span class="template-storage-cell">$_("Disk(GB)")</span> + <span class="template-storage-cell">$_("Disk Format")</span> <button type="button" id="template-edit-storage-add-button" class="action-area"></button> </div> <div class="template-tab-body"> @@ -160,6 +161,21 @@ <span class="template-storage-cell"> <input class="template-storage-disk" value={storageDisk} type="text" /> </span> + <span class="template-storage-cell"> + <input class="template-storage-disk-format" value={storageDiskFormat} type="text" style="display:none" /> + <select> + <option>qcow2</option> + <option>raw</option> + <option>bochs</option> + <option>cloop</option> + <option>cow</option> + <option>dmg</option> + <option>qcow</option> + <option>qed</option> + <option>vmdk</option> + <option>vpc</option> + </select> + </span> </div> </script> <script id="template-interface-tmpl" type="text/html"> -- 2.1.0

2 3

[PATCH V3 0/2] Warn users about NetworkManager instability
by Jose Ricardo Ziviani 27 Apr '15

27 Apr '15

V3: - Use cached capability value. V2: - Change is_nm_running to use a 'cross distro' command. - Update the warning message NetworkManager might have some instability when managing the bridges created by libvirt. There some bugs against NM about it and there is no much we can do more than warn the users about it. So, this patchset checks if NM is running and, if so, will display a message to users. Jose Ricardo Ziviani (2): Implement function to check if NM is running. Add a warn about NM running in the system. src/kimchi/model/config.py | 1 + src/kimchi/model/featuretests.py | 10 ++++++++++ tests/test_rest.py | 3 ++- ui/js/src/kimchi.network.js | 3 +++ ui/pages/i18n.json.tmpl | 1 + 5 files changed, 17 insertions(+), 1 deletion(-) -- 1.9.1

2 3

[PATCH] Allow creating VM storages with ref_cnt > 0
by Crístian Deives 27 Apr '15

27 Apr '15

As discussed in an e-mail thread (http://lists.ovirt.org/pipermail/kimchi-devel/2015-March/010154.html) we should stop blocking the user from creating VM storages when the storage has already been added to some VM. The UI may display a warning when that cases happens so the user should decide what to do. Remove check on ref_cnt when creating a VM storage. Signed-off-by: Crístian Deives <cristiandeives(a)gmail.com> --- src/kimchi/i18n.py | 1 - src/kimchi/model/vmstorages.py | 2 -- 2 files changed, 3 deletions(-) diff --git a/src/kimchi/i18n.py b/src/kimchi/i18n.py index 18e84bc..6b4726c 100644 --- a/src/kimchi/i18n.py +++ b/src/kimchi/i18n.py @@ -285,7 +285,6 @@ messages = { "KCHVMSTOR0013E": _("Specify path to update virtual machine disk"), "KCHVMSTOR0014E": _("Controller type %(type)s limitation of %(limit)s devices reached"), "KCHVMSTOR0015E": _("Cannot retrieve disk path information for given pool/volume: %(error)s"), - "KCHVMSTOR0016E": _("Volume already in use by other virtual machine."), "KCHVMSTOR0017E": _("Only one of path or pool/volume can be specified to add a new virtual machine disk"), "KCHVMSTOR0018E": _("Volume chosen with format %(format)s does not fit in the storage type %(type)s"), diff --git a/src/kimchi/model/vmstorages.py b/src/kimchi/model/vmstorages.py index 37aca64..0375772 100644 --- a/src/kimchi/model/vmstorages.py +++ b/src/kimchi/model/vmstorages.py @@ -111,8 +111,6 @@ class VMStoragesModel(object): raise InvalidParameter("KCHVMSTOR0012E") except Exception as e: raise InvalidParameter("KCHVMSTOR0015E", {'error': e}) - if vol_info['ref_cnt'] != 0: - raise InvalidParameter("KCHVMSTOR0016E") valid_format = { "disk": ["raw", "bochs", "qcow", "qcow2", "qed", "vmdk"], -- 2.1.0

2 1

[PATCH 0/3] Warn users about NetworkManager instability
by Jose Ricardo Ziviani 27 Apr '15

27 Apr '15

NetworkManager might have some instability when managing the bridges created by libvirt. There some bugs against NM about it and there is no much we can do more than warn the users about it. So, this patchset checks if NM is running and, if so, will display a message to users. Jose Ricardo Ziviani (3): Add the env parameter to run_command. Implement function to check if NM is running. Add a warn about NM running in the system. src/kimchi/model/config.py | 1 + src/kimchi/model/featuretests.py | 28 ++++++++++++++++++++++++++++ src/kimchi/utils.py | 6 ++++-- tests/test_rest.py | 3 ++- ui/js/src/kimchi.network.js | 5 +++++ ui/pages/i18n.json.tmpl | 1 + 6 files changed, 41 insertions(+), 3 deletions(-) -- 1.9.1

5 10