April 2015 - Kimchi-devel - oVirt List Archives

RFC - Guest memory usage
by Daniel Henrique Barboza 27 Oct '15

27 Oct '15

I was looking at the following item in the backlog: Guests Stats: Display memory utilization (use virt-df or virt-top or ...) If I understood it right, the idea here is to show the -inner- memory allocation of the guest. If you have a VM with 4Gb of RAM running an Ubuntu, we want to know how much memory the Ubuntu OS and its processes are using. I've done an investigation and I haven't found any tool to accomplish this. "virt-top", "virsh dommemstat" and the libvirt API retrieves the information of the memory usage of the guest relative to the host. In the example mentioned before, supposing that the host has 64Gb of RAM, all these tools would show that the VM is using 12% of the host RAM. They do not dive in the VM and shows the actual mem usage of the Ubuntu and its processes running there. Haven't found anything useful in other MLs and forums. The common answer is 'run top in a terminal inside the VM', which of course does not suit us. My question is: any thoughts about how we can implement this feature? Because I am starting to think that, in the end, this kind of info is strict to the guest OS and can't be polled from the outside. Thanks!

5 5

[RFC] Split Kimchi into Base Framework and Virtualization layers
by Lucio Correia 05 May '15

05 May '15

Hi, Gustavo (on cc) and me have been analyzing how to split Kimchi Virtualization layer form its Base web framework. We would like to leverage all infrastructure kimchi base framework provides (URI routing, error handling, i18n, authentication, API validation, plugin tabs loading) in other projects, and contribute this back to kimchi-ginger project. The general idea for the web framework is that it provides basic UI with login and tabs menu, and each plugin installed extends the menu with some tabs. We started by making kimchi a plugin, like ginger is. We are way advanced on that and already have most of kimchi's UI running as a plugin over an almost fully running base web framework. Ginger needs minimal changes, mainly on import statements. We can provide changes needed back to ginger too. Currently, we are splitting kimchi specific stuff like this way: (after renaming the dir src/kimchi to src/backend *) * we are using the name "backend" just for disambiguation from virtualization layer (plugins/kimchi). We will need to define a better name for that basic framework. One suggestion is to call it "wok", the base where kimchi is cooked. {src/backend => plugins/kimchi}/API.json | 0 {src/backend/control/vm => plugins/kimchi/control}/Makefile.am | 7 +- {src/backend/control/vm => plugins/kimchi/control}/__init__.py | 2 +- {src/backend => plugins/kimchi}/control/config.py | 0 {src/backend => plugins/kimchi}/control/cpuinfo.py | 0 {src/backend => plugins/kimchi}/control/debugreports.py | 0 {src/backend => plugins/kimchi}/control/groups.py | 0 {src/backend => plugins/kimchi}/control/host.py | 0 {src/backend => plugins/kimchi}/control/interfaces.py | 0 {src/backend => plugins/kimchi}/control/networks.py | 0 {src/backend => plugins/kimchi}/control/peers.py | 0 {src/backend => plugins/kimchi}/control/storagepools.py | 0 {src/backend => plugins/kimchi}/control/storageservers.py | 0 {src/backend => plugins/kimchi}/control/storagevolumes.py | 0 {src/backend => plugins/kimchi}/control/templates.py | 0 {src/backend => plugins/kimchi}/control/users.py | 0 {src/backend => plugins/kimchi}/control/vm/Makefile.am | 0 {src/backend => plugins/kimchi}/control/vm/__init__.py | 0 {src/backend => plugins/kimchi}/control/vm/hostdevs.py | 0 {src/backend => plugins/kimchi}/control/vm/ifaces.py | 0 {src/backend => plugins/kimchi}/control/vm/snapshots.py | 0 {src/backend => plugins/kimchi}/control/vm/storages.py | 0 {src/backend => plugins/kimchi}/control/vms.py | 0 {src/backend => plugins/kimchi}/disks.py | 0 {src/backend => plugins/kimchi}/imageinfo.py | 0 {src/backend => plugins/kimchi}/iscsi.py | 0 {src/backend => plugins/kimchi}/isoinfo.py | 0 {src/backend => plugins/kimchi}/kvmusertests.py | 0 {src/backend => plugins/kimchi}/mockmodel.py | 0 {src/backend/control/vm => plugins/kimchi/model}/Makefile.am | 7 +- {src/backend/control/vm => plugins/kimchi/model}/__init__.py | 8 --- {src/backend => plugins/kimchi}/model/config.py | 0 {src/backend => plugins/kimchi}/model/cpuinfo.py | 0 {src/backend => plugins/kimchi}/model/debugreports.py | 0 {src/backend => plugins/kimchi}/model/diskutils.py | 0 {src/backend => plugins/kimchi}/model/featuretests.py | 0 {src/backend => plugins/kimchi}/model/groups.py | 0 {src/backend => plugins/kimchi}/model/host.py | 0 {src/backend => plugins/kimchi}/model/hostdev.py | 0 {src/backend => plugins/kimchi}/model/interfaces.py | 0 {src/backend => plugins/kimchi}/model/libvirtconnection.py | 0 {src/backend => plugins/kimchi}/model/libvirtstoragepool.py | 0 plugins/kimchi/model/model.py | 93 ++++++++++++++++++++++++++++ {src/backend => plugins/kimchi}/model/networks.py | 0 {src/backend => plugins/kimchi}/model/peers.py | 0 {src/backend => plugins/kimchi}/model/storagepools.py | 0 {src/backend => plugins/kimchi}/model/storageservers.py | 0 {src/backend => plugins/kimchi}/model/storagetargets.py | 0 {src/backend => plugins/kimchi}/model/storagevolumes.py | 0 {src/backend => plugins/kimchi}/model/templates.py | 0 {src/backend => plugins/kimchi}/model/users.py | 0 {src/backend => plugins/kimchi}/model/utils.py | 0 {src/backend => plugins/kimchi}/model/vmhostdevs.py | 0 {src/backend => plugins/kimchi}/model/vmifaces.py | 0 {src/backend => plugins/kimchi}/model/vms.py | 0 {src/backend => plugins/kimchi}/model/vmsnapshots.py | 0 {src/backend => plugins/kimchi}/model/vmstorages.py | 0 {src/backend => plugins/kimchi}/osinfo.py | 0 {src/backend => plugins/kimchi}/scan.py | 0 {config => plugins/kimchi/ui}/Makefile.am | 0 {config/ui => plugins/kimchi/ui/config}/Makefile.am | 0 config/ui/tabs.xml => plugins/kimchi/ui/config/tab-ext.xml | 0 {src/backend => plugins/kimchi}/vmtemplate.py | 0 The remaining generic stuff on the "backend" side, usable or already in use by other plugins, would be: backend |___model | |___model.py | |___plugins.py | |___tasks.py |___control | |___base.py | |___plugins.py | |___tasks.py | |___utils.py |___xmlutils | |___network.py | |___disk.py | |___utils.py | |___graphics.py | |___qemucmdline.py | |___interface.py |___asynctask.py |___auth.py |___basemodel.py |___cachebust.py |___distroloader.py |___exception.py |___i18n.py |___netinfo.py |___network.py |___objectstore.py |___proxy.py |___repositories.py |___rollbackcontext.py |___root.py |___screenshot.py |___server.py |___sslcert.py |___swupdate.py |___template.py |___utils.py |___vnc.py We don't want to make this a too long e-mail and lose the focus on the idea. We have further technical details to discuss in a follow up. Please let us know your thoughts on that, any directions and requirements you have on this, and how could we proceed with that plan. -- Lucio Correia Software Engineer IBM LTC Brazil

2 3

[kimchi-devel][PATCHv4 0/7] Upload storage volume
by lvroyce＠linux.vnet.ibm.com 04 May '15

04 May '15

From: Royce Lv <lvroyce(a)linux.vnet.ibm.com> How to upload: POST /storagepools/<pool-name>/storagevolumes/ {'capacity':size-of-the-volume,'name':name-of-the-volume,'format':'raw'} PUT /storagepools/<pool-name>/storagevolumes/<volume-name> {'chunk':chunk-data,'chunk_size':size-of-chunk,'index':index-of-the-chunk-of-uploaded-file} PATCH 7/7 is a script against real model server, you can use it to verify the logic of upload. Royce Lv (7): Update docs and json schema of storage volume upload Update controller to make update accept formdata params Add lock facility for storage volume upload Update model for storage volume update Fix incomplete record when uploading update test case for storage volume upload A test script against real model server docs/API.md | 7 ++- src/kimchi/API.json | 22 ++++++++ src/kimchi/control/base.py | 6 +-- src/kimchi/i18n.py | 4 ++ src/kimchi/isoinfo.py | 5 +- src/kimchi/model/storagevolumes.py | 61 ++++++++++----------- src/kimchi/model/utils.py | 46 ++++++++++++++++ tests/test_api.py | 108 +++++++++++++++++++++++++++++++++++++ tests/test_model_storagevolume.py | 95 ++++++++++++++++++++++++-------- 9 files changed, 291 insertions(+), 63 deletions(-) create mode 100644 tests/test_api.py -- 2.1.0

3 15

[PATCH 0/5] Support vm name with special characters
by Ramon Medeiros 04 May '15

04 May '15

Ramon Medeiros (5): Create title tag Change the way that get_vm() and vm_list() search for VMs Use get_vm() instead of conn.lookupByName Encode vm_name to generate VNC token Reflect the unicode conversion on token creation src/kimchi/model/vms.py | 50 +++++++++++++++++++++++++++++++++--------- src/kimchi/model/vmstorages.py | 4 ++-- src/kimchi/vmtemplate.py | 4 +++- ui/js/src/kimchi.api.js | 4 ++-- 4 files changed, 47 insertions(+), 15 deletions(-) -- 2.1.0

2 11

[PATCH v2] Supress error messages while checking vm metadata
by Ramon Medeiros 04 May '15

04 May '15

Changes v2: Use parse functions present on model/utils.py The function FeatureTests.disable_libvirt_error_logging only removes the error message that will be displayed on the console. Libvirt also logs errors on messages log. To avoid flood messages log with metadata errors, it will be verified manually if the xml has the tag. Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> --- src/kimchi/model/utils.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/kimchi/model/utils.py b/src/kimchi/model/utils.py index b2739b2..1b114f6 100644 --- a/src/kimchi/model/utils.py +++ b/src/kimchi/model/utils.py @@ -95,6 +95,9 @@ def _kimchi_set_metadata_node(dom, node): def libvirt_get_kimchi_metadata_node(dom, mode="current"): + if _kimchi_get_metadata_node(dom, "metadata") == None: + return + FeatureTests.disable_libvirt_error_logging() try: xml = dom.metadata(libvirt.VIR_DOMAIN_METADATA_ELEMENT, -- 2.1.0

2 1

[PATCH 0/3] Snapshot enhancements
by Crístian Deives 04 May '15

04 May '15

Crístian Deives (3): snapshot: Handle non-existing snapshots in mock lookup snapshot: Change the action 'revert' to a Task snapshot: Allow creating a snapshot of a domain in any state src/kimchi/control/vm/snapshots.py | 2 +- src/kimchi/i18n.py | 1 - src/kimchi/mockmodel.py | 14 ++++++++++++++ src/kimchi/model/vmsnapshots.py | 36 ++++++++++++++++++++++-------------- tests/test_model.py | 15 +++++++++++---- tests/test_rest.py | 13 +++++++++++++ 6 files changed, 61 insertions(+), 20 deletions(-) -- 2.1.0

3 9

[PATCH] Do not resolve hostname to IP in using remote ISOs
by Aline Manera 04 May '15

04 May '15

Due a limitation in old versions of qemu-io (qemu-io was not able to resolve hostnames preveting the ISO streaming support), Kimchi automatically resolved the hostname to IP to workaround this problem. As all the most used Linux distributions (RHEl7.1, Ubuntu 14.10, Fedora 21 and openSUSE 13.2) have a working version of qemu-io, Kimchi does not need to care about it anymore. It also fixes the issue #436 Signed-off-by: Aline Manera <alinefm(a)linux.vnet.ibm.com> Signed-off-by: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com> --- src/kimchi/model/config.py | 2 -- src/kimchi/model/featuretests.py | 26 -------------------------- src/kimchi/model/utils.py | 11 ----------- src/kimchi/model/vms.py | 1 - src/kimchi/model/vmstorages.py | 10 +++------- src/kimchi/vmtemplate.py | 12 ++---------- tests/test_model.py | 11 +---------- 7 files changed, 6 insertions(+), 67 deletions(-) diff --git a/src/kimchi/model/config.py b/src/kimchi/model/config.py index a417c9b..ca0a74b 100644 --- a/src/kimchi/model/config.py +++ b/src/kimchi/model/config.py @@ -51,7 +51,6 @@ class CapabilitiesModel(object): def __init__(self, **kargs): self.conn = kargs['conn'] self.qemu_stream = False - self.qemu_stream_dns = False self.libvirt_stream_protocols = [] self.fc_host_support = False self.metadata_support = False @@ -88,7 +87,6 @@ class CapabilitiesModel(object): kimchi_log.info("*** Running feature tests ***") conn = self.conn.get() self.qemu_stream = FeatureTests.qemu_supports_iso_stream() - self.qemu_stream_dns = FeatureTests.qemu_iso_stream_dns() self.nfs_target_probe = FeatureTests.libvirt_support_nfs_probe(conn) self.fc_host_support = FeatureTests.libvirt_support_fc_host(conn) self.metadata_support = FeatureTests.has_metadata_support(conn) diff --git a/src/kimchi/model/featuretests.py b/src/kimchi/model/featuretests.py index 8cbe31d..9400151 100644 --- a/src/kimchi/model/featuretests.py +++ b/src/kimchi/model/featuretests.py @@ -21,9 +21,7 @@ import cherrypy import libvirt import lxml.etree as ET import platform -import socket import subprocess -import threading from lxml.builder import E @@ -153,30 +151,6 @@ class FeatureTests(object): return len(stderr) == 0 @staticmethod - @servermethod - def qemu_iso_stream_dns(): - host = socket.getfqdn(cherrypy.server.socket_host) - port = cherrypy.server.socket_port - cmd = ["qemu-io", "-r", "http://%s:%d/images/icon-fedora.png" % - (host, port), "-c", "'read -v 0 512'"] - proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - - thread = threading.Thread(target=proc.communicate) - thread.start() - thread.join(5) - - if thread.is_alive(): - proc.kill() - thread.join() - return False - - if proc.returncode != 0: - return False - - return True - - @staticmethod def libvirt_support_fc_host(conn): try: FeatureTests.disable_libvirt_error_logging() diff --git a/src/kimchi/model/utils.py b/src/kimchi/model/utils.py index b2739b2..0c9d519 100644 --- a/src/kimchi/model/utils.py +++ b/src/kimchi/model/utils.py @@ -18,8 +18,6 @@ # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA import libvirt -import socket -import urlparse from lxml import etree, objectify from lxml.builder import E, ElementMaker @@ -41,15 +39,6 @@ def get_vm_name(vm_name, t_name, name_list): raise OperationFailed("KCHUTILS0003E") -def check_remote_disk_path(path, qemu_stream_dns): - hostname = urlparse.urlparse(path).hostname - if hostname is not None and not qemu_stream_dns: - ip = socket.gethostbyname(hostname) - return path.replace(hostname, ip) - - return path - - def get_vm_config_flag(dom, mode="persistent"): # libvirt.VIR_DOMAIN_AFFECT_CURRENT is 0 # VIR_DOMAIN_AFFECT_LIVE is 1, VIR_DOMAIN_AFFECT_CONFIG is 2 diff --git a/src/kimchi/model/vms.py b/src/kimchi/model/vms.py index c8267c1..a6ca27b 100644 --- a/src/kimchi/model/vms.py +++ b/src/kimchi/model/vms.py @@ -125,7 +125,6 @@ class VMsModel(object): stream_protocols = self.caps.libvirt_stream_protocols xml = t.to_vm_xml(name, vm_uuid, libvirt_stream_protocols=stream_protocols, - qemu_stream_dns=self.caps.qemu_stream_dns, graphics=graphics, volumes=vol_list) diff --git a/src/kimchi/model/vmstorages.py b/src/kimchi/model/vmstorages.py index 37aca64..87c6b3d 100644 --- a/src/kimchi/model/vmstorages.py +++ b/src/kimchi/model/vmstorages.py @@ -1,7 +1,7 @@ # # Project Kimchi # -# Copyright IBM, Corp. 2014 +# Copyright IBM, Corp. 2014-2015 # # This library is free software; you can redistribute it and/or # modify it under the terms of the GNU Lesser General Public @@ -26,7 +26,7 @@ from kimchi.exception import OperationFailed from kimchi.model.config import CapabilitiesModel from kimchi.model.vms import DOM_STATE_MAP, VMModel from kimchi.model.storagevolumes import StorageVolumeModel -from kimchi.model.utils import check_remote_disk_path, get_vm_config_flag +from kimchi.model.utils import get_vm_config_flag from kimchi.osinfo import lookup from kimchi.model.diskutils import get_disk_ref_cnt, set_disk_ref_cnt from kimchi.utils import kimchi_log @@ -131,8 +131,6 @@ class VMStoragesModel(object): params['disk'] = vol_info['type'] params.update(self._get_available_bus_address(params['bus'], vm_name)) - params['path'] = check_remote_disk_path(params['path'], - self.caps.qemu_stream_dns) # Add device to VM dev, xml = get_disk_xml(params) @@ -215,9 +213,7 @@ class VMStorageModel(object): if dev_info['type'] != 'cdrom': raise InvalidOperation("KCHVMSTOR0006E") - params['path'] = check_remote_disk_path(params.get('path', ''), - self.caps.qemu_stream_dns) - + params['path'] = params.get('path', '') old_disk_path = dev_info['path'] new_disk_path = params['path'] if new_disk_path != old_disk_path: diff --git a/src/kimchi/vmtemplate.py b/src/kimchi/vmtemplate.py index ef97d83..e047228 100644 --- a/src/kimchi/vmtemplate.py +++ b/src/kimchi/vmtemplate.py @@ -18,7 +18,6 @@ # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA import os -import socket import stat import time import urlparse @@ -128,7 +127,7 @@ class VMTemplate(object): except IsoFormatError: raise InvalidParameter("KCHISO0001E", {'filename': iso}) - def _get_cdrom_xml(self, libvirt_stream_protocols, qemu_stream_dns): + def _get_cdrom_xml(self, libvirt_stream_protocols): if 'cdrom' not in self.info: return '' @@ -139,11 +138,6 @@ class VMTemplate(object): params['index'] = self.info['cdrom_index'] params['path'] = self.info['cdrom'] - hostname = urlparse.urlparse(params['path']).hostname - if hostname is not None and not qemu_stream_dns: - ip = socket.gethostbyname(hostname) - params['path'] = params['path'].replace(hostname, ip) - if self.info.get('iso_stream', False): protocol = urlparse.urlparse(params['path']).scheme if protocol not in libvirt_stream_protocols: @@ -304,10 +298,8 @@ class VMTemplate(object): graphics.update(kwargs.get('graphics', {})) params['graphics'] = get_graphics_xml(graphics) - qemu_stream_dns = kwargs.get('qemu_stream_dns', False) libvirt_stream_protocols = kwargs.get('libvirt_stream_protocols', []) - cdrom_xml = self._get_cdrom_xml(libvirt_stream_protocols, - qemu_stream_dns) + cdrom_xml = self._get_cdrom_xml(libvirt_stream_protocols) if not urlparse.urlparse(self.info.get('cdrom', "")).scheme in \ libvirt_stream_protocols and \ diff --git a/tests/test_model.py b/tests/test_model.py index 210adfd..bd195b5 100644 --- a/tests/test_model.py +++ b/tests/test_model.py @@ -23,9 +23,7 @@ import os import pwd import re import shutil -import socket import time -import urlparse import unittest import uuid @@ -564,14 +562,7 @@ class ModelTests(unittest.TestCase): {'path': valid_remote_iso_path}) cdrom_info = inst.vmstorage_lookup(vm_name, cdrom_dev) cur_cdrom_path = re.sub(":80/", '/', cdrom_info['path']) - - # As Kimchi server is not running during this test case - # CapabilitiesModel.qemu_stream_dns will be always False - # so we need to convert the hostname to IP - output = urlparse.urlparse(valid_remote_iso_path) - hostname = socket.gethostbyname(output.hostname) - url = valid_remote_iso_path.replace(output.hostname, hostname) - self.assertEquals(url, cur_cdrom_path) + self.assertEquals(valid_remote_iso_path, cur_cdrom_path) @unittest.skipUnless(utils.running_as_root(), 'Must be run as root') def test_vm_storage_provisioning(self): -- 2.1.0

2 2

[PATCH 0/7] Make Template defaults configurable
by Aline Manera 30 Apr '15

30 Apr '15

Aline Manera (7): Remove useless variable in osinfo.py Merge common_spec wiht defaults value in osinfo.py Make Template defaults configurable Add libvirt-daemon-config-network package as Kimchi dependency Verify all networks set as Template defaults prior to server start up Create option to auto create ISO pool or not on server start up Verify storage pool set as Template default prior to server starts up contrib/DEBIAN/control.in | 1 + contrib/kimchi.spec.fedora.in | 3 ++ contrib/kimchi.spec.suse.in | 3 ++ docs/README.md | 14 +++--- src/Makefile.am | 2 +- src/kimchi.conf.in | 3 ++ src/kimchi/config.py.in | 1 + src/kimchi/model/model.py | 44 +---------------- src/kimchi/model/networks.py | 61 ++++++++++------------- src/kimchi/model/storagepools.py | 66 +++++++++++++++++++++++++ src/kimchi/model/templates.py | 2 - src/kimchi/osinfo.py | 101 ++++++++++++++++++++++++++++++--------- src/kimchid.in | 3 ++ src/template.conf | 47 ++++++++++++++++++ 14 files changed, 240 insertions(+), 111 deletions(-) create mode 100644 src/template.conf -- 2.1.0

2 15

[PATCH 0/6 v7] Asyn VM creation
by Aline Manera 30 Apr '15

30 Apr '15

V6 -> V7: - Update test cases - Update UI to reuse the clone box for pending VM creation Aline Manera (1): Create VMs Asynchronously: UI Christy Perez (5): Append clone to target_uri for vm clone task Tests for new clone target_uri UI changes for new clone target_uri Create VMs asynchronously: Backend Create VMs Asynchronously: Tests src/kimchi/control/vms.py | 4 +-- src/kimchi/model/vms.py | 33 ++++++++++++++++++----- tests/test_authorization.py | 25 +++++++++++------- tests/test_mockmodel.py | 14 +++++++--- tests/test_model.py | 49 +++++++++++++++++++++++----------- tests/test_rest.py | 60 +++++++++++++++++++++++++++++++----------- tests/test_server.py | 26 ++++++++---------- ui/css/theme-default/list.css | 8 +++--- ui/js/src/kimchi.guest_main.js | 32 ++++++++++++++++++---- ui/pages/guest.html.tmpl | 4 +-- ui/pages/i18n.json.tmpl | 1 + 11 files changed, 176 insertions(+), 80 deletions(-) -- 2.1.0

2 9

kimchi Chef cookbook
by Scott Hain 29 Apr '15

29 Apr '15

Hello, I wanted to share that I've put together a Chef cookbook for running a local dev/test instance of kimchi. I am using it for my own testing and I thought it might be useful to someone else. https://github.com/opscode-cookbooks/kimchi I created for internal testing, but it could easily be modified to set up hardware or other VMs (it may work already for these use cases, but I haven't tested them). Thanks! -- *Scott Hain* — Software Engineer II 360.360.7433 – shain(a)chef.io – *my: *Linkedin <http://www.linkedin.com/> Twitter <http://www.twitter.com/elejia> *CHEF* CHEF.IO <http://chef.io> TM *chef.io* <http://www.chef.io/> Blog <http://www.chef.io/blog/> Facebook <https://www.facebook.com/getchefdotcom> Twitter <https://twitter.com/chef> Youtube <https://www.youtube.com/getchef>

2 1

[PATCH 5/6] Create VMs Asynchronously: Tests
by Christy Perez 28 Apr '15

28 Apr '15

Signed-off-by: Christy Perez <christy(a)linux.vnet.ibm.com> --- tests/test_authorization.py | 23 +++++++++++------- tests/test_mockmodel.py | 12 +++++++--- tests/test_model.py | 46 +++++++++++++++++++++++++---------- tests/test_rest.py | 58 ++++++++++++++++++++++++++++++++++----------- 4 files changed, 100 insertions(+), 39 deletions(-) diff --git a/tests/test_authorization.py b/tests/test_authorization.py index 4fcc496..eae837c 100644 --- a/tests/test_authorization.py +++ b/tests/test_authorization.py @@ -26,7 +26,7 @@ import kimchi.mockmodel from iso_gen import construct_fake_iso from utils import get_free_port, patch_auth, request -from utils import run_server +from utils import run_server, wait_task test_server = None @@ -118,19 +118,24 @@ def test_nonroot_access(self): # Non-root users can only get vms authorized to them model.templates_create({'name': u'test', 'cdrom': fake_iso}) - model.vms_create({'name': u'test-me', 'template': '/templates/test'}) + task_info = model.vms_create({'name': u'test-me', + 'template': '/templates/test'}) + wait_task(model.task_lookup, task_info['id']) + model.vm_update(u'test-me', {'users': [kimchi.mockmodel.fake_user.keys()[0]], 'groups': []}) - model.vms_create({'name': u'test-usera', - 'template': '/templates/test'}) + task_info = model.vms_create({'name': u'test-usera', + 'template': '/templates/test'}) + wait_task(model.task_lookup, task_info['id']) non_root = list(set(model.users_get_list()) - set(['root']))[0] model.vm_update(u'test-usera', {'users': [non_root], 'groups': []}) - model.vms_create({'name': u'test-groupa', - 'template': '/templates/test'}) + task_info = model.vms_create({'name': u'test-groupa', + 'template': '/templates/test'}) + wait_task(model.task_lookup, task_info['id']) a_group = model.groups_get_list()[0] model.vm_update(u'test-groupa', {'groups': [a_group]}) @@ -143,9 +148,9 @@ def test_nonroot_access(self): self.assertEquals(403, resp.status) # Create a vm using mockmodel directly to test Resource access - model.vms_create({'name': 'kimchi-test', - 'template': '/templates/test'}) - + task_info = model.vms_create({'name': 'kimchi-test', + 'template': '/templates/test'}) + wait_task(model.task_lookup, task_info['id']) resp = self.request('/vms/kimchi-test', '{}', 'PUT') self.assertEquals(403, resp.status) resp = self.request('/vms/kimchi-test', '{}', 'DELETE') diff --git a/tests/test_mockmodel.py b/tests/test_mockmodel.py index aa48dd1..a731579 100644 --- a/tests/test_mockmodel.py +++ b/tests/test_mockmodel.py @@ -66,7 +66,9 @@ def test_screenshot_refresh(self): req = json.dumps({'name': 'test', 'cdrom': fake_iso}) request(host, ssl_port, '/templates', req, 'POST') req = json.dumps({'name': 'test-vm', 'template': '/templates/test'}) - request(host, ssl_port, '/vms', req, 'POST') + resp = request(host, ssl_port, '/vms', req, 'POST') + task = json.loads(resp.read()) + wait_task(model.task_lookup, task['id']) # Test screenshot refresh for running vm request(host, ssl_port, '/vms/test-vm/start', '{}', 'POST') @@ -94,7 +96,9 @@ def test_vm_list_sorted(self): def add_vm(name): # Create a VM req = json.dumps({'name': name, 'template': '/templates/test'}) - request(host, ssl_port, '/vms', req, 'POST') + task = json.loads(request(host, ssl_port, '/vms', req, + 'POST').read()) + wait_task(model.task_lookup, task['id']) vms = [u'abc', u'bca', u'cab', u'xba'] for vm in vms: @@ -106,7 +110,9 @@ def add_vm(name): def test_vm_info(self): model.templates_create({'name': u'test', 'cdrom': fake_iso}) - model.vms_create({'name': u'test-vm', 'template': '/templates/test'}) + task = model.vms_create({'name': u'test-vm', + 'template': '/templates/test'}) + wait_task(model.task_lookup, task['id']) vms = model.vms_get_list() self.assertEquals(2, len(vms)) self.assertIn(u'test-vm', vms) diff --git a/tests/test_model.py b/tests/test_model.py index ad0dccd..7e6090e 100644 --- a/tests/test_model.py +++ b/tests/test_model.py @@ -129,8 +129,11 @@ def test_vm_lifecycle(self): rollback.prependDefer(inst.template_delete, 'test') params = {'name': 'kimchi-vm', 'template': '/templates/test'} - inst.vms_create(params) + task = inst.vms_create(params) rollback.prependDefer(inst.vm_delete, 'kimchi-vm') + inst.task_wait(task['id'], 10) + task = inst.task_lookup(task['id']) + self.assertEquals('finished', task['status']) vms = inst.vms_get_list() self.assertTrue('kimchi-vm' in vms) @@ -267,7 +270,8 @@ def test_image_based_template(self): session.store('template', tmpl_name, tmpl_info) params = {'name': 'kimchi-vm', 'template': '/templates/img-tmpl'} - inst.vms_create(params) + task = inst.vms_create(params) + inst.task_wait(task['id']) rollback.prependDefer(inst.vm_delete, 'kimchi-vm') vms = inst.vms_get_list() @@ -286,7 +290,8 @@ def test_vm_graphics(self): inst.templates_create(params) with RollbackContext() as rollback: params = {'name': 'kimchi-vnc', 'template': '/templates/test'} - inst.vms_create(params) + task1 = inst.vms_create(params) + inst.task_wait(task1['id']) rollback.prependDefer(inst.vm_delete, 'kimchi-vnc') info = inst.vm_lookup('kimchi-vnc') @@ -296,7 +301,8 @@ def test_vm_graphics(self): graphics = {'type': 'spice', 'listen': '127.0.0.1'} params = {'name': 'kimchi-spice', 'template': '/templates/test', 'graphics': graphics} - inst.vms_create(params) + task2 = inst.vms_create(params) + inst.task_wait(task2['id']) rollback.prependDefer(inst.vm_delete, 'kimchi-spice') info = inst.vm_lookup('kimchi-spice') @@ -312,6 +318,10 @@ def test_vm_ifaces(self): params = {'name': 'test', 'disks': [], 'cdrom': UBUNTU_ISO} inst.templates_create(params) rollback.prependDefer(inst.template_delete, 'test') + params = {'name': 'kimchi-ifaces', 'template': '/templates/test'} + task = inst.vms_create(params) + inst.task_wait(task['id']) + rollback.prependDefer(inst.vm_delete, 'kimchi-ifaces') # Create a network net_name = 'test-network' @@ -422,7 +432,8 @@ def _attach_disk(expect_bus=modern_disk_bus): inst.templates_create(params) rollback.prependDefer(inst.template_delete, 'test') params = {'name': vm_name, 'template': '/templates/test'} - inst.vms_create(params) + task1 = inst.vms_create(params) + inst.task_wait(task1['id']) rollback.prependDefer(inst.vm_delete, vm_name) prev_count = len(inst.vmstorages_get_list(vm_name)) @@ -465,7 +476,8 @@ def _attach_disk(expect_bus=modern_disk_bus): rollback.prependDefer(inst.template_delete, 'old_distro_template') params = {'name': vm_name, 'template': '/templates/old_distro_template'} - inst.vms_create(params) + task2 = inst.vms_create(params) + inst.task_wait(task2['id']) rollback.prependDefer(inst.vm_delete, vm_name) # Need to check the right disk_bus for old distro @@ -486,7 +498,8 @@ def test_vm_cdrom(self): inst.templates_create(params) rollback.prependDefer(inst.template_delete, 'test') params = {'name': vm_name, 'template': '/templates/test'} - inst.vms_create(params) + task = inst.vms_create(params) + inst.task_wait(task['id']) rollback.prependDefer(inst.vm_delete, vm_name) prev_count = len(inst.vmstorages_get_list(vm_name)) @@ -575,7 +588,8 @@ def test_vm_storage_provisioning(self): rollback.prependDefer(inst.template_delete, 'test') params = {'name': 'test-vm-1', 'template': '/templates/test'} - inst.vms_create(params) + task = inst.vms_create(params) + inst.task_wait(task['id']) rollback.prependDefer(inst.vm_delete, 'test-vm-1') vm_info = inst.vm_lookup(params['name']) @@ -596,10 +610,12 @@ def test_vm_edit(self): with RollbackContext() as rollback: params_1 = {'name': 'kimchi-vm1', 'template': '/templates/test'} params_2 = {'name': 'kimchi-vm2', 'template': '/templates/test'} - inst.vms_create(params_1) + task1 = inst.vms_create(params_1) + inst.task_wait(task1['id']) rollback.prependDefer(utils.rollback_wrapper, inst.vm_delete, 'kimchi-vm1') - inst.vms_create(params_2) + task2 = inst.vms_create(params_2) + inst.task_wait(task2['id']) rollback.prependDefer(utils.rollback_wrapper, inst.vm_delete, 'kimchi-vm2') @@ -778,11 +794,13 @@ def test_delete_running_vm(self): rollback.prependDefer(inst.template_delete, 'test') params = {'name': u'kīмсhī-∨м', 'template': u'/templates/test'} - inst.vms_create(params) + task = inst.vms_create(params) + inst.task_wait(task['id']) rollback.prependDefer(utils.rollback_wrapper, inst.vm_delete, u'kīмсhī-∨м') inst.vm_start(u'kīмсhī-∨м') + self.assertEquals(inst.vm_lookup(u'kīмсhī-∨м')['state'], 'running') rollback.prependDefer(utils.rollback_wrapper, inst.vm_poweroff, u'kīмсhī-∨м') @@ -801,7 +819,8 @@ def test_vm_list_sorted(self): rollback.prependDefer(inst.template_delete, 'test') params = {'name': 'kimchi-vm', 'template': '/templates/test'} - inst.vms_create(params) + task = inst.vms_create(params) + inst.task_wait(task['id']) rollback.prependDefer(inst.vm_delete, 'kimchi-vm') vms = inst.vms_get_list() @@ -873,7 +892,8 @@ def test_use_test_host(self): params = {'name': 'kimchi-vm', 'template': '/templates/test'} - inst.vms_create(params) + task = inst.vms_create(params) + inst.task_wait(task['id']) rollback.prependDefer(inst.vm_delete, 'kimchi-vm') vms = inst.vms_get_list() diff --git a/tests/test_rest.py b/tests/test_rest.py index 65c3db5..8826424 100644 --- a/tests/test_rest.py +++ b/tests/test_rest.py @@ -112,7 +112,9 @@ def test_get_vms(self): req = json.dumps({'name': name, 'template': '/templates/test', 'users': test_users, 'groups': test_groups}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) vms = json.loads(self.request('/vms').read()) self.assertEquals(11, len(vms)) @@ -130,7 +132,9 @@ def test_edit_vm(self): req = json.dumps({'name': 'vm-1', 'template': '/templates/test'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) vm = json.loads(self.request('/vms/vm-1').read()) self.assertEquals('vm-1', vm['name']) @@ -247,7 +251,9 @@ def test_vm_lifecycle(self): # Create a VM req = json.dumps({'name': 'test-vm', 'template': '/templates/test'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) + self.assertEquals(202, resp.status) # Verify the VM vm = json.loads(self.request('/vms/test-vm').read()) @@ -429,7 +435,9 @@ def test_vm_graphics(self): # Create a VM with default args req = json.dumps({'name': 'test-vm', 'template': '/templates/test'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Verify the VM vm = json.loads(self.request('/vms/test-vm').read()) self.assertEquals('127.0.0.1', vm['graphics']['listen']) @@ -443,7 +451,9 @@ def test_vm_graphics(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test', 'graphics': graphics}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Verify the VM vm = json.loads(self.request('/vms/test-vm').read()) self.assertEquals('127.0.0.1', vm['graphics']['listen']) @@ -457,7 +467,9 @@ def test_vm_graphics(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test', 'graphics': graphics}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Verify the VM vm = json.loads(self.request('/vms/test-vm').read()) self.assertEquals('fe00::0', vm['graphics']['listen']) @@ -471,7 +483,9 @@ def test_vm_graphics(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test', 'graphics': graphics}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Verify the VM vm = json.loads(self.request('/vms/test-vm').read()) self.assertEquals('127.0.0.1', vm['graphics']['listen']) @@ -513,7 +527,9 @@ def test_vm_storage_devices(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Delete the VM rollback.prependDefer(self.request, '/vms/test-vm', '{}', 'DELETE') @@ -658,7 +674,9 @@ def test_vm_iface(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Delete the VM rollback.prependDefer(self.request, '/vms/test-vm', '{}', 'DELETE') @@ -738,7 +756,10 @@ def test_vm_customise_storage(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test', 'storagepool': '/storagepools/alt'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) + resp = self.request('/vms/test-vm', {}, 'GET') vm_info = json.loads(resp.read()) # Test template not changed after vm customise its pool @@ -791,7 +812,9 @@ def test_scsi_fc_storage(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test_fc_pool'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Start the VM resp = self.request('/vms/test-vm/start', '{}', 'POST') @@ -816,8 +839,10 @@ def test_unnamed_vms(self): # Create 5 unnamed vms from this template for i in xrange(1, 6): req = json.dumps({'template': '/templates/test'}) - vm = json.loads(self.request('/vms', req, 'POST').read()) - self.assertEquals('test-vm-%i' % i, vm['name']) + task = json.loads(self.request('/vms', req, 'POST').read()) + wait_task(self._task_lookup, task['id']) + resp = self.request('/vms/test-vm-%i' % i, {}, 'GET') + self.assertEquals(resp.status, 200) count = len(json.loads(self.request('/vms').read())) self.assertEquals(6, count) @@ -849,7 +874,10 @@ def test_create_vm_with_img_based_template(self): self.assertEquals(201, resp.status) req = json.dumps({'template': '/templates/test'}) - json.loads(self.request('/vms', req, 'POST').read()) + resp = self.request('/vms', req, 'POST') + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Test storage volume created with backing store of base file resp = json.loads( @@ -939,6 +967,8 @@ def test_screenshot_refresh(self): resp = self.request('/templates', req, 'POST') req = json.dumps({'name': 'test-vm', 'template': '/templates/test'}) resp = self.request('/vms', req, 'POST') + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Test screenshot for shut-off state vm resp = self.request('/vms/test-vm/screenshot') -- 2.1.0

1 0

[PATCH 1/6] Append clone to target_uri for vm clone task
by Christy Perez 28 Apr '15

28 Apr '15

So that we can differentiate between clone and create tasks for VMs. Signed-off-by: Christy Perez <christy(a)linux.vnet.ibm.com> --- src/kimchi/model/vms.py | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/src/kimchi/model/vms.py b/src/kimchi/model/vms.py index a6ca27b..5f8c3da 100644 --- a/src/kimchi/model/vms.py +++ b/src/kimchi/model/vms.py @@ -229,9 +229,8 @@ def clone(self, name): new_name = get_next_clone_name(current_vm_names, name) # create a task with the actual clone function - taskid = add_task(u'/vms/%s' % new_name, self._clone_task, - self.objstore, - {'name': name, 'new_name': new_name}) + taskid = add_task(u'/vms/%s/clone' % new_name, self._clone_task, + self.objstore, {'name': name, 'new_name': new_name}) return self.task.lookup(taskid) -- 2.1.0

1 1

[PATCH v6 0/6] Async VM Creation
by Christy Perez 28 Apr '15

28 Apr '15

v6 Changes: - Rebase v5 Changes: - Rebase - Remove all the unimperative target_uri changes (see below) If a guest has a large disk, and uses a filesystem that requires preallocation, it can take several minutes to create a VM. During that time, kimchi is tied up by the VM creation. This patch changes the VMs Collection to be an AsyncCollection. Another change required for this was to create a more granular way to query vm-related tasks. The original idea was to add another field to the task database, but then Aline suggested just modifying the task_uri. Since the task_uri cretation will be changed in a future patchset, this is now only modified for the conflicting (clone) tasks. Christy Perez (6): Append clone to target_uri for vm clone task Tests for new clone target_uri UI changes for new clone target_uri Create VMs asynchronously: Backend Create VMs Asynchronously: Tests Create VMs Asynchronously: UI src/kimchi/control/vms.py | 4 +-- src/kimchi/model/vms.py | 32 +++++++++++++++++---- tests/test_authorization.py | 23 +++++++++------ tests/test_mockmodel.py | 12 ++++++-- tests/test_model.py | 50 ++++++++++++++++++++++---------- tests/test_model_storagevolume.py | 2 +- tests/test_rest.py | 60 +++++++++++++++++++++++++++++---------- ui/css/theme-default/list.css | 18 ++++++++++++ ui/js/src/kimchi.guest_main.js | 29 +++++++++++++++---- ui/pages/guest.html.tmpl | 3 ++ 10 files changed, 177 insertions(+), 56 deletions(-) -- 2.1.0

1 5

[PATCH 6/6] Create VMs Asynchronously: UI
by Christy Perez 28 Apr '15

28 Apr '15

Signed-off-by: Christy Perez <christy(a)linux.vnet.ibm.com> --- ui/css/theme-default/list.css | 18 ++++++++++++++++++ ui/js/src/kimchi.guest_main.js | 25 ++++++++++++++++++++++--- ui/pages/guest.html.tmpl | 3 +++ 3 files changed, 43 insertions(+), 3 deletions(-) diff --git a/ui/css/theme-default/list.css b/ui/css/theme-default/list.css index 7b32ea6..62d1539 100644 --- a/ui/css/theme-default/list.css +++ b/ui/css/theme-default/list.css @@ -306,3 +306,21 @@ margin-left: 5px; text-shadow: -1px -1px 1px #CCCCCC, 1px 1px 1px #FFFFFF; } + +.guest-create { + margin: 10px; +} + +.guest-create .icon { + background: url('../../images/theme-default/kimchi-loading15x15.gif') no-repeat; + display: inline-block; + width: 20px; + height: 20px; + vertical-align: middle; +} + +.guest-create .text { + color: #666666; + margin-left: 5px; + text-shadow: -1px -1px 1px #CCCCCC, 1px 1px 1px #FFFFFF; +} diff --git a/ui/js/src/kimchi.guest_main.js b/ui/js/src/kimchi.guest_main.js index c712bb4..dbcc162 100644 --- a/ui/js/src/kimchi.guest_main.js +++ b/ui/js/src/kimchi.guest_main.js @@ -201,6 +201,21 @@ kimchi.listVmsAuto = function() { if (kimchi.vmTimeout) { clearTimeout(kimchi.vmTimeout); } + var getCreatingGuests = function(){ + var guests = []; + kimchi.getTasksByFilter('status=running&target_uri='+encodeURIComponent('^/vms/[^/]+$'), function(tasks) { + for(var i=0;i<tasks.length;i++){ + var guestUri = tasks[i].target_uri; + var guestName = guestUri.split('/')[1] + guests.push($.extend({}, kimchi.sampleGuestObject, {name: guestName, isCreating: true})); + if(kimchi.trackingTasks.indexOf(tasks[i].id)==-1) + kimchi.trackTask(tasks[i].id, null, function(err){ + kimchi.message.error(err.message); + }, null); + } + }, null, true); + return guests; + }; var getCloningGuests = function(){ var guests = []; kimchi.getTasksByFilter('status=running&target_uri='+encodeURIComponent('^/vms/.+/clone'), function(tasks) { @@ -219,6 +234,7 @@ kimchi.listVmsAuto = function() { kimchi.listVMs(function(result, textStatus, jqXHR) { if (result && textStatus=="success") { result = getCloningGuests().concat(result); + result = getCreatingGuests().concat(result); if(result.length) { var listHtml = ''; var guestTemplate = kimchi.guestTemplate; @@ -281,7 +297,7 @@ kimchi.createGuestLi = function(vmObject, prevScreenImage, openMenu) { imgLoad.attr('src',load_src); //Link the stopped tile to the start action, the running tile to open the console - if(!vmObject.isCloning){ + if(!(vmObject.isCloning || vmObject.isCreating)){ if (vmRunningBool) { liveTile.off("click", kimchi.vmstart); liveTile.on("click", kimchi.openVmConsole); @@ -329,7 +345,7 @@ kimchi.createGuestLi = function(vmObject, prevScreenImage, openMenu) { } //Setup action event handlers - if(!vmObject.isCloning){ + if(!(vmObject.isCloning || vmObject.isCreating)){ guestActions.find("[name=vm-start]").on({click : kimchi.vmstart}); guestActions.find("[name=vm-poweroff]").on({click : kimchi.vmpoweroff}); if (vmRunningBool) { //If the guest is not running, do not enable reset @@ -362,7 +378,10 @@ kimchi.createGuestLi = function(vmObject, prevScreenImage, openMenu) { }else{ guestActions.find('.btn').attr('disabled', true); - result.find('.guest-clone').removeClass('hide-content'); + if(vmObject.isCloning) + result.find('.guest-clone').removeClass('hide-content'); + else + result.find('.guest-create').removeClass('hide-content'); $('.popover', guestActions.find("div[name=actionmenu]")).remove(); } diff --git a/ui/pages/guest.html.tmpl b/ui/pages/guest.html.tmpl index 17d41ac..aaf41a2 100644 --- a/ui/pages/guest.html.tmpl +++ b/ui/pages/guest.html.tmpl @@ -29,6 +29,9 @@ <div class="guest-clone hide-content"> <span class="icon"></span><span class="text">$_("Cloning")...</span> </div> + <div class="guest-create hide-content"> + <span class="icon"></span><span class="text">$_("Creating")...</span> + </div> </div> <div name="cpu_utilization" class="sortable"> <div class="circleGauge"></div> -- 2.1.0

1 0

[PATCH 4/6] Create VMs asynchronously: Backend
by Christy Perez 28 Apr '15

28 Apr '15

Signed-off-by: Christy Perez <christy(a)linux.vnet.ibm.com> --- src/kimchi/control/vms.py | 4 ++-- src/kimchi/model/vms.py | 27 ++++++++++++++++++++++++--- 2 files changed, 26 insertions(+), 5 deletions(-) diff --git a/src/kimchi/control/vms.py b/src/kimchi/control/vms.py index 6352a26..a40b56e 100644 --- a/src/kimchi/control/vms.py +++ b/src/kimchi/control/vms.py @@ -17,13 +17,13 @@ # License along with this library; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA -from kimchi.control.base import Collection, Resource +from kimchi.control.base import AsyncCollection, Resource from kimchi.control.utils import internal_redirect, UrlSubNode from kimchi.control.vm import sub_nodes @UrlSubNode('vms', True) -class VMs(Collection): +class VMs(AsyncCollection): def __init__(self, model): super(VMs, self).__init__(model) self.resource = VM diff --git a/src/kimchi/model/vms.py b/src/kimchi/model/vms.py index 5f8c3da..9786063 100644 --- a/src/kimchi/model/vms.py +++ b/src/kimchi/model/vms.py @@ -79,9 +79,9 @@ def __init__(self, **kargs): self.conn = kargs['conn'] self.objstore = kargs['objstore'] self.caps = CapabilitiesModel(**kargs) + self.task = TaskModel(**kargs) def create(self, params): - conn = self.conn.get() t_name = template_name_from_uri(params['template']) vm_uuid = str(uuid.uuid4()) vm_list = self.get_list() @@ -102,7 +102,26 @@ def create(self, params): raise InvalidOperation("KCHVM0005E") t.validate() + taskid = add_task(u'/vms/%s' % name, + self._create_task, self.objstore, + {'vm_uuid': vm_uuid, 'template': t, 'name': name}) + + return self.task.lookup(taskid) + + def _create_task(self, cb, params): + """ + params: A dict with the following values: + - vm_uuid: The UUID of the VM being created + - template: The template being used to create the VM + - name: The name for the new VM + """ + + vm_uuid = params['vm_uuid'] + t = params['template'] + name = params['name'] + conn = self.conn.get() + cb('Storing VM icon') # Store the icon for displaying later icon = t.info.get('icon') if icon: @@ -117,6 +136,7 @@ def create(self, params): # If storagepool is SCSI, volumes will be LUNs and must be passed by # the user from UI or manually. + cb('Provisioning storage for new VM') vol_list = [] if t._get_storage_type() not in ["iscsi", "scsi"]: vol_list = t.fork_vm_storage(vm_uuid) @@ -128,6 +148,7 @@ def create(self, params): graphics=graphics, volumes=vol_list) + cb('Defining new VM') try: conn.defineXML(xml.encode('utf-8')) except libvirt.libvirtError as e: @@ -138,10 +159,10 @@ def create(self, params): raise OperationFailed("KCHVM0007E", {'name': name, 'err': e.get_error_message()}) + cb('Updating VM metadata') VMModel.vm_update_os_metadata(VMModel.get_vm(name, self.conn), t.info, self.caps.metadata_support) - - return name + cb('OK', True) def get_list(self): return VMsModel.get_vms(self.conn) -- 2.1.0

1 0

[PATCH 3/6] UI changes for new clone target_uri
by Christy Perez 28 Apr '15

28 Apr '15

Signed-off-by: Christy Perez <christy(a)linux.vnet.ibm.com> --- ui/js/src/kimchi.guest_main.js | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ui/js/src/kimchi.guest_main.js b/ui/js/src/kimchi.guest_main.js index b66177c..c712bb4 100644 --- a/ui/js/src/kimchi.guest_main.js +++ b/ui/js/src/kimchi.guest_main.js @@ -203,10 +203,10 @@ kimchi.listVmsAuto = function() { } var getCloningGuests = function(){ var guests = []; - kimchi.getTasksByFilter('status=running&target_uri='+encodeURIComponent('^/vms/*'), function(tasks) { + kimchi.getTasksByFilter('status=running&target_uri='+encodeURIComponent('^/vms/.+/clone'), function(tasks) { for(var i=0;i<tasks.length;i++){ var guestUri = tasks[i].target_uri; - var guestName = guestUri.substring(guestUri.lastIndexOf('/')+1, guestUri.length); + var guestName = guestUri.split('/')[2] guests.push($.extend({}, kimchi.sampleGuestObject, {name: guestName, isCloning: true})); if(kimchi.trackingTasks.indexOf(tasks[i].id)==-1) kimchi.trackTask(tasks[i].id, null, function(err){ -- 2.1.0

1 0

[PATCH v5 0/6] Async VM Creation
by Christy Perez 28 Apr '15

28 Apr '15

v5 Changes: - Rebase - Remove all the unimperative target_uri changes (see below) If a guest has a large disk, and uses a filesystem that requires preallocation, it can take several minutes to create a VM. During that time, kimchi is tied up by the VM creation. This patch changes the VMs Collection to be an AsyncCollection. Another change required for this was to create a more granular way to query vm-related tasks. The original idea was to add another field to the task database, but then Aline suggested just modifying the task_uri. Since the task_uri cretation will be changed in a future patchset, this is now only modified for the conflicting (clone) tasks. Christy Perez (6): Append clone to target_uri for vm clone task Tests for new clone target_uri UI changes for new clone target_uri Create VMs asynchronously: Backend Create VMs Asynchronously: Tests Create VMs Asynchronously: UI src/kimchi/control/vms.py | 4 +-- src/kimchi/model/vms.py | 32 +++++++++++++++++---- tests/test_authorization.py | 23 +++++++++------ tests/test_mockmodel.py | 12 ++++++-- tests/test_model.py | 49 +++++++++++++++++++++----------- tests/test_model_storagevolume.py | 2 +- tests/test_rest.py | 60 +++++++++++++++++++++++++++++---------- ui/css/theme-default/list.css | 18 ++++++++++++ ui/js/src/kimchi.guest_main.js | 29 +++++++++++++++---- ui/pages/guest.html.tmpl | 3 ++ 10 files changed, 175 insertions(+), 57 deletions(-) -- 2.1.0

2 9

[PATCH] Improve code to retrieve the number of host CPUs
by Jose Ricardo Ziviani 28 Apr '15

28 Apr '15

- Retrieving the number of cpus using psinfo is unstable, there are three (at least) different ways to do it that changes from version to version. This code improves the way such functions are called to avoid any surprise in future. Signed-off-by: Jose Ricardo Ziviani <joserz(a)linux.vnet.ibm.com> --- src/kimchi/model/host.py | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/src/kimchi/model/host.py b/src/kimchi/model/host.py index 293c4ce..b2fa379 100644 --- a/src/kimchi/model/host.py +++ b/src/kimchi/model/host.py @@ -102,15 +102,20 @@ class HostModel(object): return res def lookup(self, *name): - cpus = 0 + cpus = psutil.NUM_CPUS - # Only newer psutil versions have a portable method to get - # the number of cpus - try: + # psutil is unstable on how to get the number of + # cpus, different versions call it differently + if hasattr(psutil, 'cpu_count'): cpus = psutil.cpu_count() - except AttributeError: - cpus = psutil._psplatform.get_num_cpus() + elif hasattr(psutil, '_psplatform'): + for method_name in ['_get_num_cpus', 'get_num_cpus']: + + method = getattr(psutil._psplatform, method_name, None) + if method is not None: + cpus = method() + break self.host_info['cpus'] = cpus self.host_info['memory'] = psutil.phymem_usage().total -- 1.9.1

2 2

[PATCH] Move kimchi nginx config file to nginx default directory.
by Jose Ricardo Ziviani 28 Apr '15

28 Apr '15

- Move Kimchi nginx config file to /etc/nginx/conf.d. - Rename nginx_kimchi.conf to kimchi.conf. --- .gitignore | 2 +- configure.ac | 1 + contrib/kimchi.spec.fedora.in | 6 ++-- contrib/kimchi.spec.suse.in | 7 ++-- src/Makefile.am | 5 ++- src/kimchi/config.py.in | 2 ++ src/kimchi/proxy.py | 12 +++---- src/nginx.conf.in | 76 ------------------------------------------- src/nginx/Makefile.am | 23 +++++++++++++ src/nginx/kimchi.conf.in | 76 +++++++++++++++++++++++++++++++++++++++++++ 10 files changed, 119 insertions(+), 91 deletions(-) delete mode 100644 src/nginx.conf.in create mode 100644 src/nginx/Makefile.am create mode 100644 src/nginx/kimchi.conf.in diff --git a/.gitignore b/.gitignore index 4abfc0a..a318bd9 100644 --- a/.gitignore +++ b/.gitignore @@ -28,7 +28,7 @@ stamp-po kimchi-*.tar.gz src/kimchid src/kimchi.conf -src/nginx_kimchi.conf +src/nginx/kimchi.conf src/kimchi/config.py tests/run_tests.sh tests/test_config.py diff --git a/configure.ac b/configure.ac index 1b476c3..5da3240 100644 --- a/configure.ac +++ b/configure.ac @@ -80,6 +80,7 @@ AC_CONFIG_FILES([ docs/Makefile src/Makefile src/distros.d/Makefile + src/nginx/Makefile src/kimchi/Makefile src/kimchi/control/Makefile src/kimchi/control/vm/Makefile diff --git a/contrib/kimchi.spec.fedora.in b/contrib/kimchi.spec.fedora.in index 750dada..a721a7f 100644 --- a/contrib/kimchi.spec.fedora.in +++ b/contrib/kimchi.spec.fedora.in @@ -94,7 +94,7 @@ touch %{buildroot}/%{_localstatedir}/log/kimchi/kimchi-error.log # create /etc/kimchi structure mkdir -p %{buildroot}/%{_sysconfdir}/kimchi/ -touch %{buildroot}/%{_sysconfdir}/kimchi/nginx_kimchi.conf +touch %{buildroot}/%{_sysconfdir}/nginx/conf.d/kimchi.conf # Install the systemd scripts install -Dm 0644 contrib/kimchid.service.fedora %{buildroot}%{_unitdir}/kimchid.service @@ -158,14 +158,14 @@ rm -rf $RPM_BUILD_ROOT %{_datadir}/kimchi/config/ui/*.xml %{_datadir}/kimchi/ui/ %{_datadir}/kimchi +%{_sysconfdir}/nginx/conf.d/kimchi.conf.in +%{_sysconfdir}/nginx/conf.d/kimchi.conf %{_sysconfdir}/kimchi/kimchi.conf -%{_sysconfdir}/kimchi/nginx.conf.in %{_sysconfdir}/kimchi/distros.d/debian.json %{_sysconfdir}/kimchi/distros.d/fedora.json %{_sysconfdir}/kimchi/distros.d/opensuse.json %{_sysconfdir}/kimchi/distros.d/ubuntu.json %{_sysconfdir}/kimchi/distros.d/gentoo.json -%{_sysconfdir}/kimchi/nginx_kimchi.conf %{_sysconfdir}/kimchi/ %{_sharedstatedir}/kimchi/debugreports/ %{_sharedstatedir}/kimchi/screenshots/ diff --git a/contrib/kimchi.spec.suse.in b/contrib/kimchi.spec.suse.in index 7e4172d..5673ced 100644 --- a/contrib/kimchi.spec.suse.in +++ b/contrib/kimchi.spec.suse.in @@ -63,7 +63,9 @@ touch %{buildroot}/%{_localstatedir}/log/kimchi/kimchi-error.log # create /etc/kimchi structure mkdir -p %{buildroot}/%{_sysconfdir}/kimchi/ -touch %{buildroot}/%{_sysconfdir}/kimchi/nginx_kimchi.conf + +# create nginx configuration +touch %{buildroot}/%{_sysconfdir}/nginx/conf.d/kimchi.conf # Install the SysV init scripts install -Dm 0755 contrib/kimchid.sysvinit %{buildroot}%{_initrddir}/kimchid @@ -97,13 +99,14 @@ rm -rf $RPM_BUILD_ROOT %{_datadir}/kimchi/config/ui/*.xml %{_datadir}/kimchi/ui/ %{_datadir}/kimchi +%{_sysconfdir}/nginx/conf.d/kimchi.conf.in +%{_sysconfdir}/nginx/conf.d/kimchi.conf %{_sysconfdir}/kimchi/kimchi.conf %{_sysconfdir}/kimchi/distros.d/debian.json %{_sysconfdir}/kimchi/distros.d/fedora.json %{_sysconfdir}/kimchi/distros.d/opensuse.json %{_sysconfdir}/kimchi/distros.d/ubuntu.json %{_sysconfdir}/kimchi/distros.d/gentoo.json -%{_sysconfdir}/kimchi/nginx_kimchi.conf %{_sysconfdir}/kimchi %{_initrddir}/kimchid %{_sysconfdir}/kimchi/ diff --git a/src/Makefile.am b/src/Makefile.am index dfeb24e..edc163e 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -17,18 +17,17 @@ # License along with this library; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA -SUBDIRS = kimchi distros.d +SUBDIRS = kimchi distros.d nginx EXTRA_DIST = kimchid.in \ kimchi.conf.in \ - nginx.conf.in \ firewalld.xml \ $(NULL) bin_SCRIPTS = kimchid confdir = $(sysconfdir)/kimchi -dist_conf_DATA = kimchi.conf nginx.conf.in +dist_conf_DATA = kimchi.conf BUILT_SOURCES = kimchi.conf diff --git a/src/kimchi/config.py.in b/src/kimchi/config.py.in index f2e1cac..f15d228 100644 --- a/src/kimchi/config.py.in +++ b/src/kimchi/config.py.in @@ -112,6 +112,7 @@ class Paths(object): self.novnc_dir = '/usr/share/novnc' if self.installed: + self.nginx_conf_dir = '@sysconfdir@/nginx/conf.d' self.state_dir = '@localstatedir@/lib/kimchi' self.log_dir = '@localstatedir@/log/kimchi' self.conf_dir = '@sysconfdir@/kimchi' @@ -120,6 +121,7 @@ class Paths(object): self.mo_dir = '@prefix@/share/locale' self.spice_css_file = os.path.join(self.spice_dir, 'spice.css') else: + self.nginx_conf_dir = self.add_prefix('src/nginx') self.state_dir = self.add_prefix('data') self.log_dir = self.add_prefix('log') self.conf_dir = self.add_prefix('src') diff --git a/src/kimchi/proxy.py b/src/kimchi/proxy.py index fafa5bc..1b70d5e 100644 --- a/src/kimchi/proxy.py +++ b/src/kimchi/proxy.py @@ -37,8 +37,7 @@ def _create_proxy_config(options): To allow flexibility in which port kimchi runs, we need the same flexibility with the nginx proxy. This method creates the config file dynamically by using 'nginx.conf.in' as a template, creating - the file 'nginx_kimchi.config' which will be used to launch the - proxy. + the file 'kimchi.conf' which will be used to launch the proxy. Arguments: options - OptionParser object with Kimchi config options @@ -53,6 +52,7 @@ def _create_proxy_config(options): user_proxy = 'www-data' config_dir = paths.conf_dir + nginx_config_dir = paths.nginx_conf_dir cert = options.ssl_cert key = options.ssl_key @@ -70,7 +70,7 @@ def _create_proxy_config(options): # Read template file and create a new config file # with the specified parameters. - with open(os.path.join(config_dir, "nginx.conf.in")) as template: + with open(os.path.join(nginx_config_dir, "kimchi.conf.in")) as template: data = template.read() data = Template(data) data = data.safe_substitute(user=user_proxy, @@ -81,7 +81,7 @@ def _create_proxy_config(options): max_body_size=eval(options.max_body_size)) # Write file to be used for nginx. - config_file = open(os.path.join(config_dir, "nginx_kimchi.conf"), "w") + config_file = open(os.path.join(nginx_config_dir, "kimchi.conf"), "w") config_file.write(data) config_file.close() @@ -89,8 +89,8 @@ def _create_proxy_config(options): def start_proxy(options): """Start nginx reverse proxy.""" _create_proxy_config(options) - config_dir = paths.conf_dir - config_file = "%s/nginx_kimchi.conf" % config_dir + nginx_config_dir = paths.nginx_conf_dir + config_file = "%s/kimchi.conf" % nginx_config_dir cmd = ['nginx', '-c', config_file] subprocess.call(cmd) diff --git a/src/nginx.conf.in b/src/nginx.conf.in deleted file mode 100644 index e308152..0000000 --- a/src/nginx.conf.in +++ /dev/null @@ -1,76 +0,0 @@ -# Project Kimchi -# -# Copyright IBM, Corp. 2014 -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library; if not, write to the Free Software -# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA -# 02110-1301 USA - -# This is a template file to be used to generate a nginx -# proxy config file at kimchid script. - -user ${user}; -worker_processes 1; - -error_log /var/log/nginx/error.log; - -events { - worker_connections 1024; -} - -http { - - log_format main '$remote_addr - $remote_user [$time_local] "$request" ' - '$status $body_bytes_sent "$http_referer" ' - '"$http_user_agent" "$http_x_forwarded_for"'; - - access_log /var/log/nginx/access.log main; - sendfile on; - - client_max_body_size ${max_body_size}k; - - # Timeout set to 10 minutes to avoid the 504 Gateway Timeout - # when Kimchi is processing a request. - proxy_connect_timeout 600; - proxy_send_timeout 600; - proxy_read_timeout 600; - send_timeout 600; - - server { - listen ${proxy_ssl_port} ssl; - - ssl_certificate ${cert_pem}; - ssl_certificate_key ${cert_key}; - ssl_protocols TLSv1 TLSv1.1 TLSv1.2; - ssl_ciphers ECDH@STRENGTH:DH@STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNULL; - - add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;"; - add_header X-Frame-Options DENY; - add_header X-Content-Type-Options nosniff; - add_header X-XSS-Protection "1; mode=block"; - - location / { - proxy_pass http://127.0.0.1:${kimchid_port}; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_redirect http://127.0.0.1:${kimchid_port}/ https://$host:${proxy_ssl_port}/; - } - } - - server { - listen ${proxy_port}; - rewrite ^/(.*)$ https://$host:${proxy_ssl_port}/$1 redirect; - } -} diff --git a/src/nginx/Makefile.am b/src/nginx/Makefile.am new file mode 100644 index 0000000..b240508 --- /dev/null +++ b/src/nginx/Makefile.am @@ -0,0 +1,23 @@ +# +# Kimchi +# +# Copyright IBM Corp, 2013 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +EXTRA_DIST = kimchi.conf.in + +confdir = $(sysconfdir)/nginx/conf.d +dist_conf_DATA = kimchi.conf.in diff --git a/src/nginx/kimchi.conf.in b/src/nginx/kimchi.conf.in new file mode 100644 index 0000000..e308152 --- /dev/null +++ b/src/nginx/kimchi.conf.in @@ -0,0 +1,76 @@ +# Project Kimchi +# +# Copyright IBM, Corp. 2014 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301 USA + +# This is a template file to be used to generate a nginx +# proxy config file at kimchid script. + +user ${user}; +worker_processes 1; + +error_log /var/log/nginx/error.log; + +events { + worker_connections 1024; +} + +http { + + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; + + access_log /var/log/nginx/access.log main; + sendfile on; + + client_max_body_size ${max_body_size}k; + + # Timeout set to 10 minutes to avoid the 504 Gateway Timeout + # when Kimchi is processing a request. + proxy_connect_timeout 600; + proxy_send_timeout 600; + proxy_read_timeout 600; + send_timeout 600; + + server { + listen ${proxy_ssl_port} ssl; + + ssl_certificate ${cert_pem}; + ssl_certificate_key ${cert_key}; + ssl_protocols TLSv1 TLSv1.1 TLSv1.2; + ssl_ciphers ECDH@STRENGTH:DH@STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNULL; + + add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;"; + add_header X-Frame-Options DENY; + add_header X-Content-Type-Options nosniff; + add_header X-XSS-Protection "1; mode=block"; + + location / { + proxy_pass http://127.0.0.1:${kimchid_port}; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_redirect http://127.0.0.1:${kimchid_port}/ https://$host:${proxy_ssl_port}/; + } + } + + server { + listen ${proxy_port}; + rewrite ^/(.*)$ https://$host:${proxy_ssl_port}/$1 redirect; + } +} -- 1.9.1

2 2

Re: [Kimchi-devel] [PATCH 3/5] Use get_vm() instead of conn.lookupByName
by Ramon Medeiros 28 Apr '15

28 Apr '15

Please ignore these patches On 04/28/2015 10:16 AM, Ramon Medeiros wrote: > Remove duplicated code of conn.lookupByName. > > Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> > --- > src/kimchi/model/vms.py | 2 +- > src/kimchi/model/vmstorages.py | 4 ++-- > 2 files changed, 3 insertions(+), 3 deletions(-) > > diff --git a/src/kimchi/model/vms.py b/src/kimchi/model/vms.py > index 6792447..87d63c1 100644 > --- a/src/kimchi/model/vms.py > +++ b/src/kimchi/model/vms.py > @@ -266,7 +266,7 @@ class VMModel(object): > # fetch base XML > cb('reading source VM XML') > try: > - vir_dom = vir_conn.lookupByName(name) > + vir_dom = self.get_vm(name) > flags = libvirt.VIR_DOMAIN_XML_SECURE > xml = vir_dom.XMLDesc(flags).decode('utf-8') > except libvirt.libvirtError, e: > diff --git a/src/kimchi/model/vmstorages.py b/src/kimchi/model/vmstorages.py > index 37aca64..8dabae7 100644 > --- a/src/kimchi/model/vmstorages.py > +++ b/src/kimchi/model/vmstorages.py > @@ -138,7 +138,7 @@ class VMStoragesModel(object): > dev, xml = get_disk_xml(params) > try: > conn = self.conn.get() > - dom = conn.lookupByName(vm_name) > + dom = VMModel.get_vm(vm_name, conn) > dom.attachDeviceFlags(xml, get_vm_config_flag(dom, 'all')) > except Exception as e: > raise OperationFailed("KCHVMSTOR0008E", {'error': e.message}) > @@ -173,7 +173,7 @@ class VMStorageModel(object): > > try: > bus_type = self.lookup(vm_name, dev_name)['bus'] > - dom = conn.lookupByName(vm_name) > + dom = VMModel.get_vm(vm_name, conn) > except NotFoundError: > raise > -- Ramon Nunes Medeiros Kimchi Developer Linux Technology Center Brazil IBM Systems & Technology Group Phone : +55 19 2132 7878 ramonn(a)br.ibm.com

1 0

[PATCH] Fix disk format lock and checking during template creation
by Rodrigo Trujillo 27 Apr '15

27 Apr '15

Users are able to pass the disk format (qcow, raw, etc) in disk Template information. However, Kimchi was ignoring this information and always creating QCOW2 disk images (if the storagepool is not 'LOGICAL') when it creates a VM based on a given Template. This patch fixes this problem and also moves the checking and disk format auto assignment to Template creation phase, in VMTemplate class init. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/kimchi/i18n.py | 1 + src/kimchi/vmtemplate.py | 15 +++++++++++---- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/src/kimchi/i18n.py b/src/kimchi/i18n.py index 18e84bc..0651032 100644 --- a/src/kimchi/i18n.py +++ b/src/kimchi/i18n.py @@ -154,6 +154,7 @@ messages = { "KCHTMPL0025E": _("When specifying CPU topology, VCPUs must be a product of sockets, cores, and threads."), "KCHTMPL0026E": _("When specifying CPU topology, each element must be an integer greater than zero."), "KCHTMPL0027E": _("Invalid disk image format. Valid formats: bochs, cloop, cow, dmg, qcow, qcow2, qed, raw, vmdk, vpc."), + "KCHTMPL0028E": _("Invalid disk image format. Logical, SCSI and iSCSI storage pools must use disk image format 'raw'."), "KCHPOOL0001E": _("Storage pool %(name)s already exists"), "KCHPOOL0002E": _("Storage pool %(name)s does not exist"), diff --git a/src/kimchi/vmtemplate.py b/src/kimchi/vmtemplate.py index ef97d83..84e877b 100644 --- a/src/kimchi/vmtemplate.py +++ b/src/kimchi/vmtemplate.py @@ -75,8 +75,16 @@ class VMTemplate(object): args['graphics'] = graphics self.info.update(args) - # Assign right disk format to logical and [i]scsi storagepools + # Check disk image format passed and pool type if self._get_storage_type() in ['logical', 'iscsi', 'scsi']: + paramdisks = args.get('disks') + if paramdisks is not None: + for disk in paramdisks: + fmt = disk.get('format') + if fmt is not None and fmt != 'raw': + raise InvalidParameter('KCHTMPL0028E') + # Finally add disk image format when not passed or + # using default for i, disk in enumerate(self.info['disks']): self.info['disks'][i]['format'] = 'raw' @@ -198,7 +206,6 @@ class VMTemplate(object): def to_volume_list(self, vm_uuid): storage_path = self._get_storage_path() - fmt = 'raw' if self._get_storage_type() in ['logical'] else 'qcow2' ret = [] for i, d in enumerate(self.info['disks']): index = d.get('index', i) @@ -206,11 +213,11 @@ class VMTemplate(object): info = {'name': volume, 'capacity': d['size'], - 'format': fmt, + 'format': d['format'], 'path': '%s/%s' % (storage_path, volume)} if 'logical' == self._get_storage_type() or \ - fmt not in ['qcow2', 'raw']: + info['format'] not in ['qcow2', 'raw']: info['allocation'] = info['capacity'] else: info['allocation'] = 0 -- 2.1.0

2 3

[PATCH V2] UI-Template Edit: Enable user to change disk format
by Rodrigo Trujillo 27 Apr '15

27 Apr '15

This patch adds a new field (Disk Format) in Storage tab in Template edit window. Users will then be allowed to select any disk format supported by Libvirt and Kimchi backend. The default disk format is qcow2 (first option), so, for compatibility, if the template register in objecstore does not contain the disk format information, the new field is going to set 'qcow2' automatically. For iscsi storagepools, the new field is going to be disabled and the format is going to be set automatically as 'raw', just like the backend behaves currently, avoiding errors. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- ui/css/theme-default/template-edit.css | 8 ++++-- ui/js/src/kimchi.template_edit_main.js | 52 ++++++++++++++++++++++++++++++---- ui/pages/template-edit.html.tmpl | 16 +++++++++++ 3 files changed, 68 insertions(+), 8 deletions(-) diff --git a/ui/css/theme-default/template-edit.css b/ui/css/theme-default/template-edit.css index 7abee7c..ed9f02c 100644 --- a/ui/css/theme-default/template-edit.css +++ b/ui/css/theme-default/template-edit.css @@ -95,7 +95,7 @@ #edit-template-tabs .template-storage-cell{ display: inline-block; - width: 230px; + width: 180px; } #edit-template-tabs .template-storage-cell label { @@ -114,7 +114,11 @@ } #form-template-storage .template-tab-body .template-storage-name { - width: 220px; + width: 170px; +} + +#form-template-storage .template-tab-body .template-storage-disk-format { + width: 160px; } #edit-template-tabs .template-tab-body input[readonly] { diff --git a/ui/js/src/kimchi.template_edit_main.js b/ui/js/src/kimchi.template_edit_main.js index 85f76cd..6029a95 100644 --- a/ui/js/src/kimchi.template_edit_main.js +++ b/ui/js/src/kimchi.template_edit_main.js @@ -67,7 +67,7 @@ kimchi.template_edit_main = function() { $('.template-tab-body', '#form-template-storage').append(nodeStorage); var storageOptions = ''; var scsiOptions = ''; - $('select', '#form-template-storage').find('option').remove(); + $('select:first', '#form-template-storage').find('option').remove(); $.each(result, function(index, storageEntities) { if((storageEntities.state === 'active') && (storageEntities.type != 'kimchi-iso')) { if(storageEntities.type === 'iscsi' || storageEntities.type === 'scsi') { @@ -77,7 +77,7 @@ kimchi.template_edit_main = function() { var isSlected = tmpPath === thisName ? ' selected' : ''; scsiOptions += '<option' + isSlected + '>' + tmpPath + '</option>'; }); - $('select', '#form-template-storage').append(scsiOptions); + $('select:first', '#form-template-storage').append(scsiOptions); }, function() {}); } else { var isSlected = storageEntities.name === thisName ? ' selected' : ''; @@ -85,8 +85,20 @@ kimchi.template_edit_main = function() { } } }); - $('select', '#form-template-storage').append(storageOptions); - $('select', '#form-template-storage').change(function() { + $('select:first', '#form-template-storage').append(storageOptions); + + // Set disk format + $('select:last option', '#form-template-storage').each(function() { + if ($(this).text() == storageData.storageDiskFormat) { + $(this).prop('selected', true); + } + }); + + $('select:last', '#form-template-storage').change(function() { + $('.template-storage-disk-format').val($(this).val()); + }); + + $('select:first', '#form-template-storage').change(function() { var selectedItem = $(this).parent().parent(); var tempStorageNameFull = $(this).val(); var tempName = tempStorageNameFull.split('/'); @@ -99,9 +111,25 @@ kimchi.template_edit_main = function() { kimchi.getStoragePoolVolume(tempStorageName, tempName[tempName.length-1], function(info) { volSize = info.capacity / Math.pow(1024, 3); $('.template-storage-disk', selectedItem).attr('readonly', true).val(volSize); + $('select:last option', selectedItem).each(function() { + this.selected = (this.text == 'raw'); + }); + $('select:last', selectedItem).prop('disabled', true).change(); }); + } else if (tempType === 'logical') { + $('.template-storage-disk', selectedItem).attr('readonly', false); + $('select:last option', selectedItem).each(function() { + this.selected = (this.text == 'raw'); + }); + $('select:last', selectedItem).prop('disabled', true).change(); } else { $('.template-storage-disk', selectedItem).attr('readonly', false); + if ($('select:last', selectedItem).prop('disabled') == true) { + $('select:last option', selectedItem).each(function() { + this.selected = (this.text == 'qcow2'); + }); + $('select:last', selectedItem).prop('disabled', false).change(); + } } }); }); @@ -120,7 +148,8 @@ kimchi.template_edit_main = function() { editMode : 'hide', storageName : defaultPool, storageType : defaultType, - storageDisk : diskEntities.size + storageDisk : diskEntities.size, + storageDiskFormat : diskEntities.format ? diskEntities.format : 'qcow2' } if (diskEntities.volume) { @@ -131,7 +160,17 @@ kimchi.template_edit_main = function() { nodeData.storageDisk = volSize; addStorageItem(nodeData); $('.template-storage-disk').attr('readonly', true); + $('select:last option', '#form-template-storage').each(function() { + this.selected = (this.text == 'raw'); + }); + $('select:last', '#form-template-storage').prop('disabled', true).change(); + }); + } else if (defaultType === 'logical') { + addStorageItem(storageNodeData); + $('select:last option', '#form-template-storage').each(function() { + this.selected = (this.text == 'raw'); }); + $('select:last', '#form-template-storage').prop('disabled', true).change(); } else { addStorageItem(storageNodeData); } @@ -271,7 +310,8 @@ kimchi.template_edit_main = function() { origDisks[0]['volume'] && delete origDisks[0]['volume']; origDisks[0].size = Number($('.template-storage-disk', tmpItem).val()); } - data[field] = origDisks; + origDisks[0].format = $('.template-storage-disk-format', tmpItem).val(); + data[field] = origDisks; } else if (field == 'graphics') { var type = $('#form-template-general [name="' + field + '"]').val(); diff --git a/ui/pages/template-edit.html.tmpl b/ui/pages/template-edit.html.tmpl index c7832c9..e64a30f 100644 --- a/ui/pages/template-edit.html.tmpl +++ b/ui/pages/template-edit.html.tmpl @@ -104,6 +104,7 @@ <span class="template-storage-cell">$_("Storage Pool")</span> <span class="template-storage-cell">$_("Type")</span> <span class="template-storage-cell">$_("Disk(GB)")</span> + <span class="template-storage-cell">$_("Disk Format")</span> <button type="button" id="template-edit-storage-add-button" class="action-area"></button> </div> <div class="template-tab-body"> @@ -160,6 +161,21 @@ <span class="template-storage-cell"> <input class="template-storage-disk" value={storageDisk} type="text" /> </span> + <span class="template-storage-cell"> + <input class="template-storage-disk-format" value={storageDiskFormat} type="text" style="display:none" /> + <select> + <option>qcow2</option> + <option>raw</option> + <option>bochs</option> + <option>cloop</option> + <option>cow</option> + <option>dmg</option> + <option>qcow</option> + <option>qed</option> + <option>vmdk</option> + <option>vpc</option> + </select> + </span> </div> </script> <script id="template-interface-tmpl" type="text/html"> -- 2.1.0

2 3

[PATCH V3 0/2] Warn users about NetworkManager instability
by Jose Ricardo Ziviani 27 Apr '15

27 Apr '15

V3: - Use cached capability value. V2: - Change is_nm_running to use a 'cross distro' command. - Update the warning message NetworkManager might have some instability when managing the bridges created by libvirt. There some bugs against NM about it and there is no much we can do more than warn the users about it. So, this patchset checks if NM is running and, if so, will display a message to users. Jose Ricardo Ziviani (2): Implement function to check if NM is running. Add a warn about NM running in the system. src/kimchi/model/config.py | 1 + src/kimchi/model/featuretests.py | 10 ++++++++++ tests/test_rest.py | 3 ++- ui/js/src/kimchi.network.js | 3 +++ ui/pages/i18n.json.tmpl | 1 + 5 files changed, 17 insertions(+), 1 deletion(-) -- 1.9.1

2 3

[PATCH] Allow creating VM storages with ref_cnt > 0
by Crístian Deives 27 Apr '15

27 Apr '15

As discussed in an e-mail thread (http://lists.ovirt.org/pipermail/kimchi-devel/2015-March/010154.html) we should stop blocking the user from creating VM storages when the storage has already been added to some VM. The UI may display a warning when that cases happens so the user should decide what to do. Remove check on ref_cnt when creating a VM storage. Signed-off-by: Crístian Deives <cristiandeives(a)gmail.com> --- src/kimchi/i18n.py | 1 - src/kimchi/model/vmstorages.py | 2 -- 2 files changed, 3 deletions(-) diff --git a/src/kimchi/i18n.py b/src/kimchi/i18n.py index 18e84bc..6b4726c 100644 --- a/src/kimchi/i18n.py +++ b/src/kimchi/i18n.py @@ -285,7 +285,6 @@ messages = { "KCHVMSTOR0013E": _("Specify path to update virtual machine disk"), "KCHVMSTOR0014E": _("Controller type %(type)s limitation of %(limit)s devices reached"), "KCHVMSTOR0015E": _("Cannot retrieve disk path information for given pool/volume: %(error)s"), - "KCHVMSTOR0016E": _("Volume already in use by other virtual machine."), "KCHVMSTOR0017E": _("Only one of path or pool/volume can be specified to add a new virtual machine disk"), "KCHVMSTOR0018E": _("Volume chosen with format %(format)s does not fit in the storage type %(type)s"), diff --git a/src/kimchi/model/vmstorages.py b/src/kimchi/model/vmstorages.py index 37aca64..0375772 100644 --- a/src/kimchi/model/vmstorages.py +++ b/src/kimchi/model/vmstorages.py @@ -111,8 +111,6 @@ class VMStoragesModel(object): raise InvalidParameter("KCHVMSTOR0012E") except Exception as e: raise InvalidParameter("KCHVMSTOR0015E", {'error': e}) - if vol_info['ref_cnt'] != 0: - raise InvalidParameter("KCHVMSTOR0016E") valid_format = { "disk": ["raw", "bochs", "qcow", "qcow2", "qed", "vmdk"], -- 2.1.0

2 1

[PATCH 0/3] Warn users about NetworkManager instability
by Jose Ricardo Ziviani 27 Apr '15

27 Apr '15

NetworkManager might have some instability when managing the bridges created by libvirt. There some bugs against NM about it and there is no much we can do more than warn the users about it. So, this patchset checks if NM is running and, if so, will display a message to users. Jose Ricardo Ziviani (3): Add the env parameter to run_command. Implement function to check if NM is running. Add a warn about NM running in the system. src/kimchi/model/config.py | 1 + src/kimchi/model/featuretests.py | 28 ++++++++++++++++++++++++++++ src/kimchi/utils.py | 6 ++++-- tests/test_rest.py | 3 ++- ui/js/src/kimchi.network.js | 5 +++++ ui/pages/i18n.json.tmpl | 1 + 6 files changed, 41 insertions(+), 3 deletions(-) -- 1.9.1

5 10

[PATCH] Supress error messages while checking vm metadata
by Ramon Medeiros 27 Apr '15

27 Apr '15

From: Ramon Medeiros <ramonn(a)jarvis.br.ibm.com> The function FeatureTests.disable_libvirt_error_logging only removes the error message that will be displayed on the console. Libvirt also logs errors on messages log. To avoid flood messages log with metadata errors, it will be verified manually if the xml has the tag. Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> --- src/kimchi/model/utils.py | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/kimchi/model/utils.py b/src/kimchi/model/utils.py index b2739b2..749be93 100644 --- a/src/kimchi/model/utils.py +++ b/src/kimchi/model/utils.py @@ -22,6 +22,7 @@ import socket import urlparse from lxml import etree, objectify from lxml.builder import E, ElementMaker +from xml.dom import minidom from kimchi.exception import OperationFailed from kimchi.model.featuretests import FeatureTests @@ -95,6 +96,13 @@ def _kimchi_set_metadata_node(dom, node): def libvirt_get_kimchi_metadata_node(dom, mode="current"): + # check xml for metadata tag + xml = minidom.parseString(dom.XMLDesc(0)) + + # no metadata: return + if not xml.getElementsByTagName('metadata'): + return None + FeatureTests.disable_libvirt_error_logging() try: xml = dom.metadata(libvirt.VIR_DOMAIN_METADATA_ELEMENT, -- 2.1.0

2 1

[PATCH v4 0/7] Async VM Creation
by Christy Perez 27 Apr '15

27 Apr '15

If a guest has a large disk, and uses a filesystem that requires preallocation, it can take several minutes to create a VM. During that time, kimchi is tied up by the VM creation. This patch changes the VMs Collection to be an AsyncCollection. Another change required for this was to create a more granular way to query tasks. Currently it is not possible (using the API) to query tasks for the same collection or resource type that may have different operations. For example, VM cloning is also an asynchronous operation. For the guests tab, the UI was querying all running tasks and displaying them with the Cloning label. This picked up VMs that were being created as well. For more information about how the tasks can be queried, see the updated API doc and the UI change. Christy Perez (7): Granular Task Queries: Backend Granular task query test updates Granular Task Queries: UI Create guests asynchronously: Backend Async vm creation test updates VMs Async Create: UI Translation update to add 'Creating' docs/API.md | 16 ++++++ po/de_DE.po | 103 ++++++++++++++++++++----------------- po/en_US.po | 83 +++++++++++++++++------------- po/es_ES.po | 102 +++++++++++++++++++----------------- po/fr_FR.po | 102 +++++++++++++++++++----------------- po/it_IT.po | 102 +++++++++++++++++++----------------- po/ja_JP.po | 102 +++++++++++++++++++----------------- po/kimchi.pot | 84 +++++++++++++++++------------- po/ko_KR.po | 102 +++++++++++++++++++----------------- po/pt_BR.po | 102 +++++++++++++++++++----------------- po/ru_RU.po | 102 +++++++++++++++++++----------------- po/zh_CN.po | 102 +++++++++++++++++++----------------- po/zh_TW.po | 102 +++++++++++++++++++----------------- src/kimchi/control/vms.py | 4 +- src/kimchi/mockmodel.py | 7 +-- src/kimchi/model/debugreports.py | 4 +- src/kimchi/model/host.py | 4 +- src/kimchi/model/storagepools.py | 2 +- src/kimchi/model/storagevolumes.py | 12 +++-- src/kimchi/model/vms.py | 33 +++++++++--- src/kimchi/model/vmsnapshots.py | 5 +- tests/test_authorization.py | 23 +++++---- tests/test_mockmodel.py | 12 +++-- tests/test_model.py | 60 ++++++++++++++------- tests/test_model_storagevolume.py | 2 +- tests/test_rest.py | 77 ++++++++++++++++++++------- ui/css/theme-default/list.css | 18 +++++++ ui/js/src/kimchi.guest_main.js | 29 +++++++++-- ui/pages/guest.html.tmpl | 3 ++ 29 files changed, 886 insertions(+), 613 deletions(-) -- 2.1.0

4 22

[PATCH V2 0/2] Warn users about NetworkManager instability
by Jose Ricardo Ziviani 27 Apr '15

27 Apr '15

V2: - Change is_nm_running to use a 'cross distro' command. - Update the warning message NetworkManager might have some instability when managing the bridges created by libvirt. There some bugs against NM about it and there is no much we can do more than warn the users about it. So, this patchset checks if NM is running and, if so, will display a message to users. Jose Ricardo Ziviani (2): Implement function to check if NM is running. Add a warn about NM running in the system. src/kimchi/model/config.py | 1 + src/kimchi/model/featuretests.py | 10 ++++++++++ tests/test_rest.py | 3 ++- ui/js/src/kimchi.network.js | 5 +++++ ui/pages/i18n.json.tmpl | 1 + 5 files changed, 19 insertions(+), 1 deletion(-) -- 1.9.1

3 6

[PATCH] Host tests
by Aline Manera 27 Apr '15

27 Apr '15

This patch creates a new test file (test_host.py) to handle all the Host Resource related tests. It also updates the docs/API.md to reflect the right data set returned for /host/partitions API. Signed-off-by: Aline Manera <alinefm(a)linux.vnet.ibm.com> --- docs/API.md | 1 + src/kimchi/mockmodel.py | 8 +- tests/test_host.py | 192 ++++++++++++++++++++++++++++++++++++++++++++++++ tests/test_mockmodel.py | 17 ----- tests/test_model.py | 73 ------------------ tests/test_rest.py | 72 ------------------ 6 files changed, 200 insertions(+), 163 deletions(-) create mode 100644 tests/test_host.py diff --git a/docs/API.md b/docs/API.md index 3f7925f..922ded1 100644 --- a/docs/API.md +++ b/docs/API.md @@ -985,6 +985,7 @@ stats history * size: The total size of the partition, in bytes * mountpoint: If the partition is mounted, represents the mountpoint. Otherwise blank. + * available: false, if the partition is in use by system; true, otherwise. ### Collection: Devices diff --git a/src/kimchi/mockmodel.py b/src/kimchi/mockmodel.py index 413ac5d..3ebddca 100644 --- a/src/kimchi/mockmodel.py +++ b/src/kimchi/mockmodel.py @@ -42,7 +42,7 @@ from kimchi.model.templates import LibvirtVMTemplate from kimchi.model.users import PAMUsersModel from kimchi.model.groups import PAMGroupsModel from kimchi.objectstore import ObjectStore -from kimchi.utils import add_task, get_next_clone_name +from kimchi.utils import add_task, get_next_clone_name, kimchi_log from kimchi.vmtemplate import VMTemplate from kimchi.xmlutils.utils import xml_item_update @@ -273,6 +273,12 @@ class MockModel(Model): conn.storagePoolDefineXML(ET.tostring(root), 0) + def _mock_host_shutdown(self, *name): + kimchi_log.info("The host system will be shutted down") + + def _mock_host_reboot(self, *name): + kimchi_log.info("The host system will be rebooted") + def _mock_storagevolumes_create(self, pool, params): vol_source = ['file', 'url', 'capacity'] index_list = list(i for i in range(len(vol_source)) diff --git a/tests/test_host.py b/tests/test_host.py new file mode 100644 index 0000000..1273457 --- /dev/null +++ b/tests/test_host.py @@ -0,0 +1,192 @@ +# -*- coding: utf-8 -*- +# +# Project Kimchi +# +# Copyright IBM, Corp. 2015 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +import json +import os +import platform +import psutil +import tempfile +import time +import unittest + +from functools import partial + +from kimchi.mockmodel import MockModel +from utils import get_free_port, patch_auth, request, run_server, wait_task + +test_server = None +model = None +host = None +ssl_port = None +tmpfile = None + + +def setUpModule(): + global test_server, model, host, ssl_port, tmpfile + + patch_auth() + tmpfile = tempfile.mktemp() + model = MockModel(tmpfile) + host = '127.0.0.1' + port = get_free_port('http') + ssl_port = get_free_port('https') + cherrypy_port = get_free_port('cherrypy_port') + test_server = run_server(host, port, ssl_port, test_mode=True, + cherrypy_port=cherrypy_port, model=model) + + +def tearDownModule(): + test_server.stop() + os.unlink(tmpfile) + + +class HostTests(unittest.TestCase): + def setUp(self): + self.request = partial(request, host, ssl_port) + + def test_hostinfo(self): + resp = self.request('/host').read() + info = json.loads(resp) + keys = ['os_distro', 'os_version', 'os_codename', 'cpu_model', + 'memory', 'cpus'] + self.assertEquals(sorted(keys), sorted(info.keys())) + + distro, version, codename = platform.linux_distribution() + self.assertEquals(distro, info['os_distro']) + self.assertEquals(version, info['os_version']) + self.assertEquals(unicode(codename, "utf-8"), info['os_codename']) + self.assertEquals(psutil.TOTAL_PHYMEM, info['memory']) + + def test_hoststats(self): + time.sleep(1) + stats_keys = ['cpu_utilization', 'memory', 'disk_read_rate', + 'disk_write_rate', 'net_recv_rate', 'net_sent_rate'] + resp = self.request('/host/stats').read() + stats = json.loads(resp) + self.assertEquals(sorted(stats_keys), sorted(stats.keys())) + + cpu_utilization = stats['cpu_utilization'] + self.assertIsInstance(cpu_utilization, float) + self.assertGreaterEqual(cpu_utilization, 0.0) + self.assertTrue(cpu_utilization <= 100.0) + + memory_stats = stats['memory'] + self.assertIn('total', memory_stats) + self.assertIn('free', memory_stats) + self.assertIn('cached', memory_stats) + self.assertIn('buffers', memory_stats) + self.assertIn('avail', memory_stats) + + resp = self.request('/host/stats/history').read() + history = json.loads(resp) + self.assertEquals(sorted(stats_keys), sorted(history.keys())) + + def test_host_actions(self): + def _task_lookup(taskid): + return json.loads(self.request('/tasks/%s' % taskid).read()) + + resp = self.request('/host/shutdown', '{}', 'POST') + self.assertEquals(200, resp.status) + resp = self.request('/host/reboot', '{}', 'POST') + self.assertEquals(200, resp.status) + + # Test system update + resp = self.request('/host/packagesupdate', None, 'GET') + pkgs = json.loads(resp.read()) + self.assertEquals(3, len(pkgs)) + + pkg_keys = ['package_name', 'repository', 'arch', 'version'] + for p in pkgs: + name = p['package_name'] + resp = self.request('/host/packagesupdate/' + name, None, 'GET') + info = json.loads(resp.read()) + self.assertEquals(sorted(pkg_keys), sorted(info.keys())) + + resp = self.request('/host/swupdate', '{}', 'POST') + task = json.loads(resp.read()) + task_params = [u'id', u'message', u'status', u'target_uri'] + self.assertEquals(sorted(task_params), sorted(task.keys())) + + resp = self.request('/tasks/' + task[u'id'], None, 'GET') + task_info = json.loads(resp.read()) + self.assertEquals(task_info['status'], 'running') + wait_task(_task_lookup, task_info['id']) + resp = self.request('/tasks/' + task[u'id'], None, 'GET') + task_info = json.loads(resp.read()) + self.assertEquals(task_info['status'], 'finished') + self.assertIn(u'All packages updated', task_info['message']) + pkgs = model.packagesupdate_get_list() + self.assertEquals(0, len(pkgs)) + + def test_host_partitions(self): + resp = self.request('/host/partitions') + self.assertEquals(200, resp.status) + partitions = json.loads(resp.read()) + + keys = ['name', 'path', 'type', 'fstype', 'size', 'mountpoint', + 'available'] + for item in partitions: + resp = self.request('/host/partitions/%s' % item['name']) + info = json.loads(resp.read()) + self.assertEquals(sorted(info.keys()), sorted(keys)) + + def test_host_devices(self): + def asset_devices_type(devices, dev_type): + for dev in devices: + self.assertEquals(dev['device_type'], dev_type) + + resp = self.request('/host/devices?_cap=scsi_host') + nodedevs = json.loads(resp.read()) + # Mockmodel brings 3 preconfigured scsi fc_host + self.assertEquals(3, len(nodedevs)) + + nodedev = json.loads(self.request('/host/devices/scsi_host2').read()) + # Mockmodel generates random wwpn and wwnn + self.assertEquals('scsi_host2', nodedev['name']) + self.assertEquals('fc_host', nodedev['adapter']['type']) + self.assertEquals(16, len(nodedev['adapter']['wwpn'])) + self.assertEquals(16, len(nodedev['adapter']['wwnn'])) + + devs = json.loads(self.request('/host/devices').read()) + dev_names = [dev['name'] for dev in devs] + for dev_type in ('pci', 'usb_device', 'scsi'): + resp = self.request('/host/devices?_cap=%s' % dev_type) + devsByType = json.loads(resp.read()) + names = [dev['name'] for dev in devsByType] + self.assertTrue(set(names) <= set(dev_names)) + asset_devices_type(devsByType, dev_type) + + resp = self.request('/host/devices?_passthrough=true') + passthru_devs = [dev['name'] for dev in json.loads(resp.read())] + self.assertTrue(set(passthru_devs) <= set(dev_names)) + + for dev_type in ('pci', 'usb_device', 'scsi'): + resp = self.request('/host/devices?_cap=%s&_passthrough=true' % + dev_type) + filteredDevs = json.loads(resp.read()) + filteredNames = [dev['name'] for dev in filteredDevs] + self.assertTrue(set(filteredNames) <= set(dev_names)) + asset_devices_type(filteredDevs, dev_type) + + for dev in passthru_devs: + resp = self.request('/host/devices?_passthrough_affected_by=%s' % + dev) + affected_devs = [dev['name'] for dev in json.loads(resp.read())] + self.assertTrue(set(affected_devs) <= set(dev_names)) diff --git a/tests/test_mockmodel.py b/tests/test_mockmodel.py index c7e733e..aa48dd1 100644 --- a/tests/test_mockmodel.py +++ b/tests/test_mockmodel.py @@ -26,7 +26,6 @@ import unittest import kimchi.mockmodel from utils import get_free_port, patch_auth, request, run_server -from utils import wait_task from kimchi.osinfo import get_template_default @@ -131,19 +130,3 @@ class MockModelTests(unittest.TestCase): self.assertEquals(stats_keys, set(info['stats'].keys())) self.assertEquals('vnc', info['graphics']['type']) self.assertEquals('127.0.0.1', info['graphics']['listen']) - - def test_packages_update(self): - pkgs = model.packagesupdate_get_list() - self.assertEquals(3, len(pkgs)) - - for pkg_name in pkgs: - pkgupdate = model.packageupdate_lookup(pkg_name) - self.assertIn('package_name', pkgupdate.keys()) - self.assertIn('repository', pkgupdate.keys()) - self.assertIn('arch', pkgupdate.keys()) - self.assertIn('version', pkgupdate.keys()) - - task = model.host_swupdate() - task_params = [u'id', u'message', u'status', u'target_uri'] - self.assertEquals(sorted(task_params), sorted(task.keys())) - wait_task(model.task_lookup, task['id']) diff --git a/tests/test_model.py b/tests/test_model.py index c25929f..210adfd 100644 --- a/tests/test_model.py +++ b/tests/test_model.py @@ -20,8 +20,6 @@ import grp import os -import platform -import psutil import pwd import re import shutil @@ -725,37 +723,6 @@ class ModelTests(unittest.TestCase): self.assertIn('ipaddr', iface) self.assertIn('netmask', iface) - @unittest.skipUnless(utils.running_as_root(), 'Must be run as root') - def test_get_devices(self): - def asset_devices_type(devices, dev_type): - for dev in devices: - self.assertEquals(dev['device_type'], dev_type) - - inst = model.Model('qemu:///system', - objstore_loc=self.tmp_store) - - devs = inst.devices_get_list() - - for dev_type in ('pci', 'usb_device', 'scsi'): - names = inst.devices_get_list(_cap=dev_type) - self.assertTrue(set(names) <= set(devs)) - infos = [inst.device_lookup(name) for name in names] - asset_devices_type(infos, dev_type) - - passthru_devs = inst.devices_get_list(_passthrough='true') - self.assertTrue(set(passthru_devs) <= set(devs)) - - for dev_type in ('pci', 'usb_device', 'scsi'): - names = inst.devices_get_list(_cap=dev_type, _passthrough='true') - self.assertTrue(set(names) <= set(devs)) - infos = [inst.device_lookup(name) for name in names] - asset_devices_type(infos, dev_type) - - for dev_name in passthru_devs: - affected_devs = inst.devices_get_list( - _passthrough_affected_by=dev_name) - self.assertTrue(set(affected_devs) <= set(devs)) - def test_async_tasks(self): class task_except(Exception): pass @@ -983,46 +950,6 @@ class ModelTests(unittest.TestCase): self.assertIn('path', distro) @unittest.skipUnless(utils.running_as_root(), 'Must be run as root') - def test_get_hostinfo(self): - inst = model.Model(None, - objstore_loc=self.tmp_store) - info = inst.host_lookup() - distro, version, codename = platform.linux_distribution() - self.assertIn('cpu_model', info) - self.assertIn('cpus', info) - self.assertEquals(distro, info['os_distro']) - self.assertEquals(version, info['os_version']) - self.assertEquals(unicode(codename, "utf-8"), info['os_codename']) - self.assertEquals(psutil.TOTAL_PHYMEM, info['memory']) - - def test_get_hoststats(self): - inst = model.Model('test:///default', - objstore_loc=self.tmp_store) - time.sleep(1.5) - stats = inst.hoststats_lookup() - stats_keys = ['cpu_utilization', 'memory', 'disk_read_rate', - 'disk_write_rate', 'net_recv_rate', 'net_sent_rate'] - self.assertEquals(sorted(stats_keys), sorted(stats.keys())) - cpu_utilization = stats['cpu_utilization'] - # cpu_utilization is set int 0, after first stats sample - # the cpu_utilization is float in range [0.0, 100.0] - self.assertIsInstance(cpu_utilization, float) - self.assertGreaterEqual(cpu_utilization, 0.0) - self.assertTrue(cpu_utilization <= 100.0) - - memory_stats = stats['memory'] - self.assertIn('total', memory_stats) - self.assertIn('free', memory_stats) - self.assertIn('cached', memory_stats) - self.assertIn('buffers', memory_stats) - self.assertIn('avail', memory_stats) - - history = inst.hoststatshistory_lookup() - self.assertEquals(sorted(stats_keys), sorted(history.keys())) - for key, value in history.iteritems(): - self.assertEquals(type(value), list) - - @unittest.skipUnless(utils.running_as_root(), 'Must be run as root') def test_deep_scan(self): inst = model.Model(None, objstore_loc=self.tmp_store) diff --git a/tests/test_rest.py b/tests/test_rest.py index ee350b2..4ade722 100644 --- a/tests/test_rest.py +++ b/tests/test_rest.py @@ -93,19 +93,6 @@ class RestTests(unittest.TestCase): resp = self.request(*args) self.assertEquals(code, resp.status) - def test_host_devices(self): - resp = self.request('/host/devices?_cap=scsi_host') - nodedevs = json.loads(resp.read()) - # Mockmodel brings 3 preconfigured scsi fc_host - self.assertEquals(3, len(nodedevs)) - - nodedev = json.loads(self.request('/host/devices/scsi_host2').read()) - # Mockmodel generates random wwpn and wwnn - self.assertEquals('scsi_host2', nodedev['name']) - self.assertEquals('fc_host', nodedev['adapter']['type']) - self.assertEquals(16, len(nodedev['adapter']['wwpn'])) - self.assertEquals(16, len(nodedev['adapter']['wwnn'])) - def test_get_vms(self): vms = json.loads(self.request('/vms').read()) # test_rest.py uses MockModel() which connects to libvirt URI @@ -1129,65 +1116,6 @@ class RestTests(unittest.TestCase): resp = request(host, ssl_port, debugre['uri']) self.assertEquals(200, resp.status) - def test_host(self): - resp = self.request('/host').read() - info = json.loads(resp) - - keys = ['os_distro', 'os_version', 'os_codename', 'cpu_model', - 'memory', 'cpus'] - self.assertEquals(sorted(keys), sorted(info.keys())) - - def test_hoststats(self): - stats_keys = ['cpu_utilization', 'memory', 'disk_read_rate', - 'disk_write_rate', 'net_recv_rate', 'net_sent_rate'] - resp = self.request('/host/stats').read() - stats = json.loads(resp) - self.assertEquals(sorted(stats_keys), sorted(stats.keys())) - - cpu_utilization = stats['cpu_utilization'] - self.assertIsInstance(cpu_utilization, float) - self.assertGreaterEqual(cpu_utilization, 0.0) - self.assertTrue(cpu_utilization <= 100.0) - - memory_stats = stats['memory'] - self.assertIn('total', memory_stats) - self.assertIn('free', memory_stats) - self.assertIn('cached', memory_stats) - self.assertIn('buffers', memory_stats) - self.assertIn('avail', memory_stats) - - resp = self.request('/host/stats/history').read() - history = json.loads(resp) - self.assertEquals(sorted(stats_keys), sorted(history.keys())) - - def test_packages_update(self): - resp = self.request('/host/packagesupdate', None, 'GET') - pkgs = json.loads(resp.read()) - self.assertEquals(3, len(pkgs)) - - for p in pkgs: - name = p['package_name'] - resp = self.request('/host/packagesupdate/' + name, None, 'GET') - info = json.loads(resp.read()) - self.assertIn('package_name', info.keys()) - self.assertIn('repository', info.keys()) - self.assertIn('arch', info.keys()) - self.assertIn('version', info.keys()) - - resp = self.request('/host/swupdate', '{}', 'POST') - task = json.loads(resp.read()) - task_params = [u'id', u'message', u'status', u'target_uri'] - self.assertEquals(sorted(task_params), sorted(task.keys())) - - resp = self.request('/tasks/' + task[u'id'], None, 'GET') - task_info = json.loads(resp.read()) - self.assertEquals(task_info['status'], 'running') - wait_task(self._task_lookup, task_info['id']) - resp = self.request('/tasks/' + task[u'id'], None, 'GET') - task_info = json.loads(resp.read()) - self.assertEquals(task_info['status'], 'finished') - self.assertIn(u'All packages updated', task_info['message']) - def test_repositories(self): def verify_repo(t, res): for field in ('repo_id', 'enabled', 'baseurl', 'config'): -- 2.1.0

2 2

[PATCH] Move kimchi nginx config file to nginx default directory.
by Jose Ricardo Ziviani 23 Apr '15

23 Apr '15

- Move Kimchi nginx config file to /etc/nginx/conf.d. - Rename nginx_kimchi.conf to kimchi.conf. Signed-off-by: Jose Ricardo Ziviani <joserz(a)linux.vnet.ibm.com> --- .gitignore | 2 +- configure.ac | 1 + contrib/kimchi.spec.fedora.in | 6 ++-- contrib/kimchi.spec.suse.in | 7 ++-- src/Makefile.am | 5 ++- src/kimchi/config.py.in | 2 ++ src/kimchi/proxy.py | 12 +++---- src/nginx.conf.in | 76 ------------------------------------------- src/nginx/Makefile.am | 23 +++++++++++++ src/nginx/kimchi.conf.in | 76 +++++++++++++++++++++++++++++++++++++++++++ 10 files changed, 119 insertions(+), 91 deletions(-) delete mode 100644 src/nginx.conf.in create mode 100644 src/nginx/Makefile.am create mode 100644 src/nginx/kimchi.conf.in diff --git a/.gitignore b/.gitignore index 4abfc0a..a318bd9 100644 --- a/.gitignore +++ b/.gitignore @@ -28,7 +28,7 @@ stamp-po kimchi-*.tar.gz src/kimchid src/kimchi.conf -src/nginx_kimchi.conf +src/nginx/kimchi.conf src/kimchi/config.py tests/run_tests.sh tests/test_config.py diff --git a/configure.ac b/configure.ac index 1b476c3..5da3240 100644 --- a/configure.ac +++ b/configure.ac @@ -80,6 +80,7 @@ AC_CONFIG_FILES([ docs/Makefile src/Makefile src/distros.d/Makefile + src/nginx/Makefile src/kimchi/Makefile src/kimchi/control/Makefile src/kimchi/control/vm/Makefile diff --git a/contrib/kimchi.spec.fedora.in b/contrib/kimchi.spec.fedora.in index 750dada..a721a7f 100644 --- a/contrib/kimchi.spec.fedora.in +++ b/contrib/kimchi.spec.fedora.in @@ -94,7 +94,7 @@ touch %{buildroot}/%{_localstatedir}/log/kimchi/kimchi-error.log # create /etc/kimchi structure mkdir -p %{buildroot}/%{_sysconfdir}/kimchi/ -touch %{buildroot}/%{_sysconfdir}/kimchi/nginx_kimchi.conf +touch %{buildroot}/%{_sysconfdir}/nginx/conf.d/kimchi.conf # Install the systemd scripts install -Dm 0644 contrib/kimchid.service.fedora %{buildroot}%{_unitdir}/kimchid.service @@ -158,14 +158,14 @@ rm -rf $RPM_BUILD_ROOT %{_datadir}/kimchi/config/ui/*.xml %{_datadir}/kimchi/ui/ %{_datadir}/kimchi +%{_sysconfdir}/nginx/conf.d/kimchi.conf.in +%{_sysconfdir}/nginx/conf.d/kimchi.conf %{_sysconfdir}/kimchi/kimchi.conf -%{_sysconfdir}/kimchi/nginx.conf.in %{_sysconfdir}/kimchi/distros.d/debian.json %{_sysconfdir}/kimchi/distros.d/fedora.json %{_sysconfdir}/kimchi/distros.d/opensuse.json %{_sysconfdir}/kimchi/distros.d/ubuntu.json %{_sysconfdir}/kimchi/distros.d/gentoo.json -%{_sysconfdir}/kimchi/nginx_kimchi.conf %{_sysconfdir}/kimchi/ %{_sharedstatedir}/kimchi/debugreports/ %{_sharedstatedir}/kimchi/screenshots/ diff --git a/contrib/kimchi.spec.suse.in b/contrib/kimchi.spec.suse.in index 7e4172d..5673ced 100644 --- a/contrib/kimchi.spec.suse.in +++ b/contrib/kimchi.spec.suse.in @@ -63,7 +63,9 @@ touch %{buildroot}/%{_localstatedir}/log/kimchi/kimchi-error.log # create /etc/kimchi structure mkdir -p %{buildroot}/%{_sysconfdir}/kimchi/ -touch %{buildroot}/%{_sysconfdir}/kimchi/nginx_kimchi.conf + +# create nginx configuration +touch %{buildroot}/%{_sysconfdir}/nginx/conf.d/kimchi.conf # Install the SysV init scripts install -Dm 0755 contrib/kimchid.sysvinit %{buildroot}%{_initrddir}/kimchid @@ -97,13 +99,14 @@ rm -rf $RPM_BUILD_ROOT %{_datadir}/kimchi/config/ui/*.xml %{_datadir}/kimchi/ui/ %{_datadir}/kimchi +%{_sysconfdir}/nginx/conf.d/kimchi.conf.in +%{_sysconfdir}/nginx/conf.d/kimchi.conf %{_sysconfdir}/kimchi/kimchi.conf %{_sysconfdir}/kimchi/distros.d/debian.json %{_sysconfdir}/kimchi/distros.d/fedora.json %{_sysconfdir}/kimchi/distros.d/opensuse.json %{_sysconfdir}/kimchi/distros.d/ubuntu.json %{_sysconfdir}/kimchi/distros.d/gentoo.json -%{_sysconfdir}/kimchi/nginx_kimchi.conf %{_sysconfdir}/kimchi %{_initrddir}/kimchid %{_sysconfdir}/kimchi/ diff --git a/src/Makefile.am b/src/Makefile.am index dfeb24e..edc163e 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -17,18 +17,17 @@ # License along with this library; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA -SUBDIRS = kimchi distros.d +SUBDIRS = kimchi distros.d nginx EXTRA_DIST = kimchid.in \ kimchi.conf.in \ - nginx.conf.in \ firewalld.xml \ $(NULL) bin_SCRIPTS = kimchid confdir = $(sysconfdir)/kimchi -dist_conf_DATA = kimchi.conf nginx.conf.in +dist_conf_DATA = kimchi.conf BUILT_SOURCES = kimchi.conf diff --git a/src/kimchi/config.py.in b/src/kimchi/config.py.in index f2e1cac..f15d228 100644 --- a/src/kimchi/config.py.in +++ b/src/kimchi/config.py.in @@ -112,6 +112,7 @@ class Paths(object): self.novnc_dir = '/usr/share/novnc' if self.installed: + self.nginx_conf_dir = '@sysconfdir@/nginx/conf.d' self.state_dir = '@localstatedir@/lib/kimchi' self.log_dir = '@localstatedir@/log/kimchi' self.conf_dir = '@sysconfdir@/kimchi' @@ -120,6 +121,7 @@ class Paths(object): self.mo_dir = '@prefix@/share/locale' self.spice_css_file = os.path.join(self.spice_dir, 'spice.css') else: + self.nginx_conf_dir = self.add_prefix('src/nginx') self.state_dir = self.add_prefix('data') self.log_dir = self.add_prefix('log') self.conf_dir = self.add_prefix('src') diff --git a/src/kimchi/proxy.py b/src/kimchi/proxy.py index fafa5bc..1b70d5e 100644 --- a/src/kimchi/proxy.py +++ b/src/kimchi/proxy.py @@ -37,8 +37,7 @@ def _create_proxy_config(options): To allow flexibility in which port kimchi runs, we need the same flexibility with the nginx proxy. This method creates the config file dynamically by using 'nginx.conf.in' as a template, creating - the file 'nginx_kimchi.config' which will be used to launch the - proxy. + the file 'kimchi.conf' which will be used to launch the proxy. Arguments: options - OptionParser object with Kimchi config options @@ -53,6 +52,7 @@ def _create_proxy_config(options): user_proxy = 'www-data' config_dir = paths.conf_dir + nginx_config_dir = paths.nginx_conf_dir cert = options.ssl_cert key = options.ssl_key @@ -70,7 +70,7 @@ def _create_proxy_config(options): # Read template file and create a new config file # with the specified parameters. - with open(os.path.join(config_dir, "nginx.conf.in")) as template: + with open(os.path.join(nginx_config_dir, "kimchi.conf.in")) as template: data = template.read() data = Template(data) data = data.safe_substitute(user=user_proxy, @@ -81,7 +81,7 @@ def _create_proxy_config(options): max_body_size=eval(options.max_body_size)) # Write file to be used for nginx. - config_file = open(os.path.join(config_dir, "nginx_kimchi.conf"), "w") + config_file = open(os.path.join(nginx_config_dir, "kimchi.conf"), "w") config_file.write(data) config_file.close() @@ -89,8 +89,8 @@ def _create_proxy_config(options): def start_proxy(options): """Start nginx reverse proxy.""" _create_proxy_config(options) - config_dir = paths.conf_dir - config_file = "%s/nginx_kimchi.conf" % config_dir + nginx_config_dir = paths.nginx_conf_dir + config_file = "%s/kimchi.conf" % nginx_config_dir cmd = ['nginx', '-c', config_file] subprocess.call(cmd) diff --git a/src/nginx.conf.in b/src/nginx.conf.in deleted file mode 100644 index e308152..0000000 --- a/src/nginx.conf.in +++ /dev/null @@ -1,76 +0,0 @@ -# Project Kimchi -# -# Copyright IBM, Corp. 2014 -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library; if not, write to the Free Software -# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA -# 02110-1301 USA - -# This is a template file to be used to generate a nginx -# proxy config file at kimchid script. - -user ${user}; -worker_processes 1; - -error_log /var/log/nginx/error.log; - -events { - worker_connections 1024; -} - -http { - - log_format main '$remote_addr - $remote_user [$time_local] "$request" ' - '$status $body_bytes_sent "$http_referer" ' - '"$http_user_agent" "$http_x_forwarded_for"'; - - access_log /var/log/nginx/access.log main; - sendfile on; - - client_max_body_size ${max_body_size}k; - - # Timeout set to 10 minutes to avoid the 504 Gateway Timeout - # when Kimchi is processing a request. - proxy_connect_timeout 600; - proxy_send_timeout 600; - proxy_read_timeout 600; - send_timeout 600; - - server { - listen ${proxy_ssl_port} ssl; - - ssl_certificate ${cert_pem}; - ssl_certificate_key ${cert_key}; - ssl_protocols TLSv1 TLSv1.1 TLSv1.2; - ssl_ciphers ECDH@STRENGTH:DH@STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNULL; - - add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;"; - add_header X-Frame-Options DENY; - add_header X-Content-Type-Options nosniff; - add_header X-XSS-Protection "1; mode=block"; - - location / { - proxy_pass http://127.0.0.1:${kimchid_port}; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_redirect http://127.0.0.1:${kimchid_port}/ https://$host:${proxy_ssl_port}/; - } - } - - server { - listen ${proxy_port}; - rewrite ^/(.*)$ https://$host:${proxy_ssl_port}/$1 redirect; - } -} diff --git a/src/nginx/Makefile.am b/src/nginx/Makefile.am new file mode 100644 index 0000000..b240508 --- /dev/null +++ b/src/nginx/Makefile.am @@ -0,0 +1,23 @@ +# +# Kimchi +# +# Copyright IBM Corp, 2013 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +EXTRA_DIST = kimchi.conf.in + +confdir = $(sysconfdir)/nginx/conf.d +dist_conf_DATA = kimchi.conf.in diff --git a/src/nginx/kimchi.conf.in b/src/nginx/kimchi.conf.in new file mode 100644 index 0000000..e308152 --- /dev/null +++ b/src/nginx/kimchi.conf.in @@ -0,0 +1,76 @@ +# Project Kimchi +# +# Copyright IBM, Corp. 2014 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301 USA + +# This is a template file to be used to generate a nginx +# proxy config file at kimchid script. + +user ${user}; +worker_processes 1; + +error_log /var/log/nginx/error.log; + +events { + worker_connections 1024; +} + +http { + + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; + + access_log /var/log/nginx/access.log main; + sendfile on; + + client_max_body_size ${max_body_size}k; + + # Timeout set to 10 minutes to avoid the 504 Gateway Timeout + # when Kimchi is processing a request. + proxy_connect_timeout 600; + proxy_send_timeout 600; + proxy_read_timeout 600; + send_timeout 600; + + server { + listen ${proxy_ssl_port} ssl; + + ssl_certificate ${cert_pem}; + ssl_certificate_key ${cert_key}; + ssl_protocols TLSv1 TLSv1.1 TLSv1.2; + ssl_ciphers ECDH@STRENGTH:DH@STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNULL; + + add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;"; + add_header X-Frame-Options DENY; + add_header X-Content-Type-Options nosniff; + add_header X-XSS-Protection "1; mode=block"; + + location / { + proxy_pass http://127.0.0.1:${kimchid_port}; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_redirect http://127.0.0.1:${kimchid_port}/ https://$host:${proxy_ssl_port}/; + } + } + + server { + listen ${proxy_port}; + rewrite ^/(.*)$ https://$host:${proxy_ssl_port}/$1 redirect; + } +} -- 1.9.1

1 0

[kimchi-devel][PATCH 0/6] Pool on existent VG
by lvroyce＠linux.vnet.ibm.com 23 Apr '15

23 Apr '15

From: Royce Lv <lvroyce(a)linux.vnet.ibm.com> Royce Lv (6): Storagepool on existent VG: Update document Add helper function to retrieve vg information Update controller Update model for existent VG Update mockmodel tests Fix fh leak in run_command subprocess docs/API.md | 20 ++++++++++++++++ src/kimchi/control/host.py | 6 +++++ src/kimchi/i18n.py | 2 ++ src/kimchi/mockmodel.py | 9 ++++++++ src/kimchi/model/libvirtstoragepool.py | 11 +++++---- src/kimchi/model/storagepools.py | 28 +++++++++++++---------- src/kimchi/model/vgs.py | 42 ++++++++++++++++++++++++++++++++++ src/kimchi/utils.py | 22 ++++++++++++++++-- tests/test_mock_storagepool.py | 9 +++++++- 9 files changed, 130 insertions(+), 19 deletions(-) create mode 100644 src/kimchi/model/vgs.py -- 2.1.0

2 7

[RFC] Guest Network Boot support [Part1] - Create template and guest with support to Direct Boot
by Rodrigo Trujillo 20 Apr '15

20 Apr '15

Alan Evangelista (https://github.com/alanoe) and I are willing to implement the support to Network Boot to Kimchi guests. This is the first approach [Part1] we are going to develop: The direct boot When finished, we are going to send [Part2] that will cover DHCP/TFTP, etc. ------- This feature aims to develop Kimchi's support to create Guest that will boot over Network using Libvirt Direct Boot Libvirt details: https://libvirt.org/formatdomain.html#elementsOSKernel Kimchi related bug: https://github.com/kimchi-project/kimchi/issues/372 We see three major changes in Kimchi code: 1- Change Template creation and update (BACKEND) 2- Change Template creation and update (FRONTEND) 3- Change Guest creation (BACKEND) Details: 1- Change Template creation and update (BACKEND): Backend is going to wait for these new parameters (json) in Template creation or update: "netboot": { "kernel": "<KERNEL URL>", "initrd": "<INITRD URL>", "kernelargs": "<COMMAND LINE FOR KERNEL>", "repository": "<PACKAGE REPOSITORY>" (optional) } Steps: src/kimchi/API.json -> change "templates_create" and "template_update" to receive above parameters; src/kimchi/model/templates.py -> change "TemplatesModel.create()" to check if "netboot" comes to save data properly -> change "TemplateModel.update()" to check if "netboot" comes and update data properly; doc/API.md -> change to reflect new parameters; src/kimchi/i18n.py -> add new errors if needed; Other: - Change templates auto tests if necessary; - Change "VMTemplate" class in "src/kimchi/vmtemplate.py" in order to probe DISTRO and VERSION; 2- Change Template creation and update (FRONTEND): Front-end is going to have following changes: - Add new entry in 'Add Template' window: "Network Direct Installation"; - Create new window to allow user to provide new four parameters; - Change Template update window. Add new tab to change or add network installation parameters; 3- Change Guest creation (BACKEND) Here it is necessary to change the XML creation, so we need to: src/kimchi/vmtemplate.py -> change "VMTemplate.to_vm_xml()" to check existence of parameter "netboot", then change <os> element properly. Other: - Change guests auto tests if necessary; - Change xml (VMTemplate) auto test if necessary; Rodrigo Trujillo

3 3

[PATCH v2 0/4] issue #548: Hotplug network interfaces
by Crístian Deives 20 Apr '15

20 Apr '15

Here is the difference between this and the previous patchset: - Patches 1, 2 and 3 have been added; they're just bug fixes, not exactly related to the issue #548. Patch 4 remains the same. Crístian Deives (4): Parse osinfo.lookup return parameters correctly Handle missing parameter "network" when attaching a NIC Use default network model when attaching a NIC issue #548: Hotplug network interfaces src/kimchi/i18n.py | 1 - src/kimchi/model/vmifaces.py | 57 +++++++++++++++++--------- src/kimchi/xmlutils/interface.py | 15 ++++++- tests/test_model.py | 86 ++++++++++++++++++++-------------------- tests/test_rest.py | 5 +++ 5 files changed, 100 insertions(+), 64 deletions(-) -- 2.1.0

2 6

[PATCH v2] Create empty files on rpm build
by Ramon Medeiros 20 Apr '15

20 Apr '15

Changes v2: Only remove nginx conm Issue #496 - System leftovers after uninstalling Kimchi A rpm does not remove files that are not created by the installation. So, all the files were provided by the package and now they can be uninstalled. Beyond this, the function that creates the proxy conf now accepts empty files to fill the configuration. Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> --- contrib/kimchi.spec.fedora.in | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/contrib/kimchi.spec.fedora.in b/contrib/kimchi.spec.fedora.in index 9f7bb59..750dada 100644 --- a/contrib/kimchi.spec.fedora.in +++ b/contrib/kimchi.spec.fedora.in @@ -92,6 +92,10 @@ mkdir -p %{buildroot}/%{_localstatedir}/log/kimchi/ touch %{buildroot}/%{_localstatedir}/log/kimchi/kimchi-access.log touch %{buildroot}/%{_localstatedir}/log/kimchi/kimchi-error.log +# create /etc/kimchi structure +mkdir -p %{buildroot}/%{_sysconfdir}/kimchi/ +touch %{buildroot}/%{_sysconfdir}/kimchi/nginx_kimchi.conf + # Install the systemd scripts install -Dm 0644 contrib/kimchid.service.fedora %{buildroot}%{_unitdir}/kimchid.service install -Dm 0640 src/firewalld.xml %{buildroot}%{_prefix}/lib/firewalld/services/kimchid.xml @@ -161,6 +165,7 @@ rm -rf $RPM_BUILD_ROOT %{_sysconfdir}/kimchi/distros.d/opensuse.json %{_sysconfdir}/kimchi/distros.d/ubuntu.json %{_sysconfdir}/kimchi/distros.d/gentoo.json +%{_sysconfdir}/kimchi/nginx_kimchi.conf %{_sysconfdir}/kimchi/ %{_sharedstatedir}/kimchi/debugreports/ %{_sharedstatedir}/kimchi/screenshots/ -- 1.8.3.1

4 6

[PATCH v3 0/2] Remove nginx configuration on rpm -e
by Ramon Medeiros 20 Apr '15

20 Apr '15

Changes v3: Extend changes to suse v2: Remove only nginx conf. Let certificates on the machine Ramon Medeiros (2): Create empty files on rpm build Remove nginx conf on suse contrib/kimchi.spec.fedora.in | 5 +++++ contrib/kimchi.spec.suse.in | 7 ++++++- 2 files changed, 11 insertions(+), 1 deletion(-) -- 1.8.3.1

2 3

[PATCH 1/2 V2] Set qcow2 as default disk format in new templates
by Rodrigo Trujillo 20 Apr '15

20 Apr '15

Users are allowed to provide disk format when creating a new Template. However this is optional. This option sets qcow2 as default disk format for every new Template created. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/kimchi/osinfo.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/kimchi/osinfo.py b/src/kimchi/osinfo.py index 3c31346..ede2541 100644 --- a/src/kimchi/osinfo.py +++ b/src/kimchi/osinfo.py @@ -33,7 +33,8 @@ SUPPORTED_ARCHS = {'x86': ('i386', 'i686', 'x86_64'), 'ppc64le': ('ppc64le')} -common_spec = {'cpus': 1, 'memory': 1024, 'disks': [{'index': 0, 'size': 10}], +common_spec = {'cpus': 1, 'memory': 1024, 'disks': [{'index': 0, 'size': 10, + 'format': 'qcow2'}], 'cdrom_bus': 'ide', 'cdrom_index': 2, 'mouse_bus': 'ps2'} -- 2.1.0

2 4

[PATCH V2] Template: Assign 'raw' to disk format if pool is 'logical' or [i]scsi
by Rodrigo Trujillo 20 Apr '15

20 Apr '15

Kimchi always creates raw disk format to new vm disks if the storagepool is logical, iscsi or scsi. But, new templates where not using this logic and was possible, through API calls, to create a template with a SCSI storagepool and disk format vmdk, for instance. This patch fixes this issue and makes necessary change in one of the tests in order to avoid an error with _get_storage_type() and deactivated storagepools. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/kimchi/model/templates.py | 4 ++-- src/kimchi/vmtemplate.py | 5 +++++ tests/test_template.py | 2 ++ 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/src/kimchi/model/templates.py b/src/kimchi/model/templates.py index 6bc8aca..8a1820d 100644 --- a/src/kimchi/model/templates.py +++ b/src/kimchi/model/templates.py @@ -94,7 +94,7 @@ class TemplatesModel(object): # Creates the template class with necessary information # Checkings will be done while creating this class, so any exception # will be raised here - t = LibvirtVMTemplate(params, scan=True) + t = LibvirtVMTemplate(params, scan=True, conn=self.conn) name = params['name'] try: with self.objstore as session: @@ -231,8 +231,8 @@ class TemplateModel(object): class LibvirtVMTemplate(VMTemplate): def __init__(self, args, scan=False, conn=None): - VMTemplate.__init__(self, args, scan) self.conn = conn + VMTemplate.__init__(self, args, scan) def _storage_validate(self): pool_uri = self.info['storagepool'] diff --git a/src/kimchi/vmtemplate.py b/src/kimchi/vmtemplate.py index 62772e5..3e4630d 100644 --- a/src/kimchi/vmtemplate.py +++ b/src/kimchi/vmtemplate.py @@ -75,6 +75,11 @@ class VMTemplate(object): args['graphics'] = graphics self.info.update(args) + # Assign right disk format to logical and [i]scsi storagepools + if self._get_storage_type() in ['logical', 'iscsi', 'scsi']: + for i, disk in enumerate(self.info['disks']): + self.info['disks'][i]['format'] = 'raw' + def _get_os_info(self, args, scan): distro = version = 'unknown' diff --git a/tests/test_template.py b/tests/test_template.py index 23a24fb..acdb7c6 100644 --- a/tests/test_template.py +++ b/tests/test_template.py @@ -329,6 +329,8 @@ class TemplateTests(unittest.TestCase): pool = {'type': 'dir', 'name': 'dir-pool', 'path': '/tmp/dir-pool'} self.request('/storagepools', json.dumps(pool), 'POST') + pool_uri = '/storagepools/%s' % pool['name'].encode('utf-8') + self.request(pool_uri + '/activate', '{}', 'POST') # Create a template using the custom network and pool t = {'name': 'test', 'cdrom': '/tmp/mock.iso', -- 2.1.0

2 2

[PATCH V2] Fix disk format lock during VM creation
by Rodrigo Trujillo 20 Apr '15

20 Apr '15

Users are able to pass the disk format (qcow, raw, etc) in disk Template information. However, Kimchi is ignoring this information and always creating QCOW2 disk images (if the storagepool is not 'LOGICAL') when it creates a VM based on a given Template. This patch fixes this problem. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/kimchi/vmtemplate.py | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/src/kimchi/vmtemplate.py b/src/kimchi/vmtemplate.py index ec477dd..62772e5 100644 --- a/src/kimchi/vmtemplate.py +++ b/src/kimchi/vmtemplate.py @@ -193,7 +193,6 @@ class VMTemplate(object): def to_volume_list(self, vm_uuid): storage_path = self._get_storage_path() - fmt = 'raw' if self._get_storage_type() in ['logical'] else 'qcow2' ret = [] for i, d in enumerate(self.info['disks']): index = d.get('index', i) @@ -201,11 +200,16 @@ class VMTemplate(object): info = {'name': volume, 'capacity': d['size'], - 'format': fmt, + 'format': d.get('format', 'qcow2'), 'path': '%s/%s' % (storage_path, volume)} + # Validate disk format + if self._get_storage_type() in ['logical'] and \ + info['format'] != 'raw': + info['format'] = 'raw' + if 'logical' == self._get_storage_type() or \ - fmt not in ['qcow2', 'raw']: + info['format'] not in ['qcow2', 'raw']: info['allocation'] = info['capacity'] else: info['allocation'] = 0 -- 2.1.0

2 2

[RFC]Puppet deployment for Kimchi
by Royce Lv 20 Apr '15

20 Apr '15

Hi, Kimchi's initial design concept is good out of box experience for users. With features added, we have some advanced feature which needs complex user set up (such as iscsi, nfs, ldap,peer discovery etc). This makes testing, concept model set up, configuration not easy for new users. To boost user experience, I would suggest to supply puppet script samples for these set ups.

2 2

[PATCH v4 0/7] Async Disk Creation
by Christy Perez 17 Apr '15

17 Apr '15

If a guest has a large disk, and uses a filesystem that requires preallocation, it can take several minutes to create a VM. During that time, kimchi is tied up by the VM creation. This patch changes the VMs Collection to be an AsyncCollection. Another change required for this was to create a more granular way to query tasks. Currently it is not possible (using the API) to query tasks for the same collection or resource type that may have different operations. For example, VM cloning is also an asynchronous operation. For the guests tab, the UI was querying all running tasks and displaying them with the Cloning label. This picked up VMs that were being created as well. For more information about how the tasks can be queried, see the updated API doc and the UI change. Christy Perez (7): Granular Task Queries: Backend Granular task query test updates Granular Task Queries: UI Create guests asynchronously: Backend Async vm creation test updates VMs Async Create: UI Translation update to add 'Creating' docs/API.md | 16 ++++++ po/de_DE.po | 103 ++++++++++++++++++++----------------- po/en_US.po | 83 +++++++++++++++++------------- po/es_ES.po | 102 +++++++++++++++++++----------------- po/fr_FR.po | 102 +++++++++++++++++++----------------- po/it_IT.po | 102 +++++++++++++++++++----------------- po/ja_JP.po | 102 +++++++++++++++++++----------------- po/kimchi.pot | 84 +++++++++++++++++------------- po/ko_KR.po | 102 +++++++++++++++++++----------------- po/pt_BR.po | 102 +++++++++++++++++++----------------- po/ru_RU.po | 102 +++++++++++++++++++----------------- po/zh_CN.po | 102 +++++++++++++++++++----------------- po/zh_TW.po | 102 +++++++++++++++++++----------------- src/kimchi/control/vms.py | 4 +- src/kimchi/mockmodel.py | 7 +-- src/kimchi/model/debugreports.py | 4 +- src/kimchi/model/host.py | 4 +- src/kimchi/model/storagepools.py | 2 +- src/kimchi/model/storagevolumes.py | 12 +++-- src/kimchi/model/vms.py | 33 +++++++++--- src/kimchi/model/vmsnapshots.py | 5 +- tests/test_authorization.py | 23 +++++---- tests/test_mockmodel.py | 12 +++-- tests/test_model.py | 60 ++++++++++++++------- tests/test_model_storagevolume.py | 2 +- tests/test_rest.py | 77 ++++++++++++++++++++------- ui/css/theme-default/list.css | 18 +++++++ ui/js/src/kimchi.guest_main.js | 29 +++++++++-- ui/pages/guest.html.tmpl | 3 ++ 29 files changed, 886 insertions(+), 613 deletions(-) -- 2.1.0

1 8

[PATCH] Template: Assign 'raw' to disk format if pool is 'logical' or [i]scsi
by Rodrigo Trujillo 16 Apr '15

16 Apr '15

Kimchi always creates raw disk format to new vm disks if the storagepool is logical, iscsi or scsi. But, new templates where not using this logic and was possible, through API calls, to create a template with a SCSI storagepool and disk format vmdk, for instance. This patch fixes this issue. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/kimchi/model/templates.py | 4 ++-- src/kimchi/vmtemplate.py | 5 +++++ 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/src/kimchi/model/templates.py b/src/kimchi/model/templates.py index 6bc8aca..8a1820d 100644 --- a/src/kimchi/model/templates.py +++ b/src/kimchi/model/templates.py @@ -94,7 +94,7 @@ class TemplatesModel(object): # Creates the template class with necessary information # Checkings will be done while creating this class, so any exception # will be raised here - t = LibvirtVMTemplate(params, scan=True) + t = LibvirtVMTemplate(params, scan=True, conn=self.conn) name = params['name'] try: with self.objstore as session: @@ -231,8 +231,8 @@ class TemplateModel(object): class LibvirtVMTemplate(VMTemplate): def __init__(self, args, scan=False, conn=None): - VMTemplate.__init__(self, args, scan) self.conn = conn + VMTemplate.__init__(self, args, scan) def _storage_validate(self): pool_uri = self.info['storagepool'] diff --git a/src/kimchi/vmtemplate.py b/src/kimchi/vmtemplate.py index 62772e5..a309357 100644 --- a/src/kimchi/vmtemplate.py +++ b/src/kimchi/vmtemplate.py @@ -75,6 +75,11 @@ class VMTemplate(object): args['graphics'] = graphics self.info.update(args) + # Assign right disk format to logical and [i]scsi storagepools + if self._get_storage_type() in ['logical', 'iscsi', 'scsi']: + for i,disk in enumerate(self.info['disks']): + self.info['disks'][i]['format'] = 'raw' + def _get_os_info(self, args, scan): distro = version = 'unknown' -- 2.1.0

1 1

[PATCH] UI-Template Edit: Enable user to change disk format
by Rodrigo Trujillo 16 Apr '15

16 Apr '15

This patch adds a new field (Disk Format) in Storage tab in Template edit window. Users will then be allowed to select any disk format supported by Libvirt and Kimchi backend. The default disk format is qcow2 (first option), so, for compatibility, if the template register in objecstore does not contain the disk format information, the new field is going to set 'qcow2' automatically. For iscsi storagepools, the new field is going to be disabled and the format is going to be set automatically as 'raw', just like the backend behaves currently, avoiding errors. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- ui/css/theme-default/template-edit.css | 8 +++++-- ui/js/src/kimchi.template_edit_main.js | 40 +++++++++++++++++++++++++++++----- ui/pages/template-edit.html.tmpl | 16 ++++++++++++++ 3 files changed, 56 insertions(+), 8 deletions(-) diff --git a/ui/css/theme-default/template-edit.css b/ui/css/theme-default/template-edit.css index 7abee7c..ed9f02c 100644 --- a/ui/css/theme-default/template-edit.css +++ b/ui/css/theme-default/template-edit.css @@ -95,7 +95,7 @@ #edit-template-tabs .template-storage-cell{ display: inline-block; - width: 230px; + width: 180px; } #edit-template-tabs .template-storage-cell label { @@ -114,7 +114,11 @@ } #form-template-storage .template-tab-body .template-storage-name { - width: 220px; + width: 170px; +} + +#form-template-storage .template-tab-body .template-storage-disk-format { + width: 160px; } #edit-template-tabs .template-tab-body input[readonly] { diff --git a/ui/js/src/kimchi.template_edit_main.js b/ui/js/src/kimchi.template_edit_main.js index 85f76cd..48ae26b 100644 --- a/ui/js/src/kimchi.template_edit_main.js +++ b/ui/js/src/kimchi.template_edit_main.js @@ -67,7 +67,7 @@ kimchi.template_edit_main = function() { $('.template-tab-body', '#form-template-storage').append(nodeStorage); var storageOptions = ''; var scsiOptions = ''; - $('select', '#form-template-storage').find('option').remove(); + $('select:first', '#form-template-storage').find('option').remove(); $.each(result, function(index, storageEntities) { if((storageEntities.state === 'active') && (storageEntities.type != 'kimchi-iso')) { if(storageEntities.type === 'iscsi' || storageEntities.type === 'scsi') { @@ -77,7 +77,7 @@ kimchi.template_edit_main = function() { var isSlected = tmpPath === thisName ? ' selected' : ''; scsiOptions += '<option' + isSlected + '>' + tmpPath + '</option>'; }); - $('select', '#form-template-storage').append(scsiOptions); + $('select:first', '#form-template-storage').append(scsiOptions); }, function() {}); } else { var isSlected = storageEntities.name === thisName ? ' selected' : ''; @@ -85,8 +85,20 @@ kimchi.template_edit_main = function() { } } }); - $('select', '#form-template-storage').append(storageOptions); - $('select', '#form-template-storage').change(function() { + $('select:first', '#form-template-storage').append(storageOptions); + + // Set disk format + $('select:last option', '#form-template-storage').each(function() { + if ($(this).text() == storageData.storageDiskFormat) { + $(this).prop('selected', true); + } + }); + + $('select:last', '#form-template-storage').change(function() { + $('.template-storage-disk-format').val($(this).val()); + }); + + $('select:first', '#form-template-storage').change(function() { var selectedItem = $(this).parent().parent(); var tempStorageNameFull = $(this).val(); var tempName = tempStorageNameFull.split('/'); @@ -99,9 +111,19 @@ kimchi.template_edit_main = function() { kimchi.getStoragePoolVolume(tempStorageName, tempName[tempName.length-1], function(info) { volSize = info.capacity / Math.pow(1024, 3); $('.template-storage-disk', selectedItem).attr('readonly', true).val(volSize); + $('select:last option', selectedItem).each(function() { + this.selected = (this.text == 'raw'); + }); + $('select:last', selectedItem).prop('disabled', true).change(); }); } else { $('.template-storage-disk', selectedItem).attr('readonly', false); + if ($('select:last', selectedItem).prop('disabled') == true) { + $('select:last option', selectedItem).each(function() { + this.selected = (this.text == 'qcow2'); + }); + $('select:last', selectedItem).prop('disabled', false).change(); + } } }); }); @@ -120,7 +142,8 @@ kimchi.template_edit_main = function() { editMode : 'hide', storageName : defaultPool, storageType : defaultType, - storageDisk : diskEntities.size + storageDisk : diskEntities.size, + storageDiskFormat : diskEntities.format ? diskEntities.format : 'qcow2' } if (diskEntities.volume) { @@ -131,6 +154,10 @@ kimchi.template_edit_main = function() { nodeData.storageDisk = volSize; addStorageItem(nodeData); $('.template-storage-disk').attr('readonly', true); + $('select:last option', '#form-template-storage').each(function() { + this.selected = (this.text == 'raw'); + }); + $('select:last', '#form-template-storage').prop('disabled', true).change(); }); } else { addStorageItem(storageNodeData); @@ -271,7 +298,8 @@ kimchi.template_edit_main = function() { origDisks[0]['volume'] && delete origDisks[0]['volume']; origDisks[0].size = Number($('.template-storage-disk', tmpItem).val()); } - data[field] = origDisks; + origDisks[0].format = $('.template-storage-disk-format', tmpItem).val(); + data[field] = origDisks; } else if (field == 'graphics') { var type = $('#form-template-general [name="' + field + '"]').val(); diff --git a/ui/pages/template-edit.html.tmpl b/ui/pages/template-edit.html.tmpl index c7832c9..e64a30f 100644 --- a/ui/pages/template-edit.html.tmpl +++ b/ui/pages/template-edit.html.tmpl @@ -104,6 +104,7 @@ <span class="template-storage-cell">$_("Storage Pool")</span> <span class="template-storage-cell">$_("Type")</span> <span class="template-storage-cell">$_("Disk(GB)")</span> + <span class="template-storage-cell">$_("Disk Format")</span> <button type="button" id="template-edit-storage-add-button" class="action-area"></button> </div> <div class="template-tab-body"> @@ -160,6 +161,21 @@ <span class="template-storage-cell"> <input class="template-storage-disk" value={storageDisk} type="text" /> </span> + <span class="template-storage-cell"> + <input class="template-storage-disk-format" value={storageDiskFormat} type="text" style="display:none" /> + <select> + <option>qcow2</option> + <option>raw</option> + <option>bochs</option> + <option>cloop</option> + <option>cow</option> + <option>dmg</option> + <option>qcow</option> + <option>qed</option> + <option>vmdk</option> + <option>vpc</option> + </select> + </span> </div> </script> <script id="template-interface-tmpl" type="text/html"> -- 2.1.0

2 2

[PATCH] Set qcow2 as default disk format in new templates
by Rodrigo Trujillo 16 Apr '15

16 Apr '15

Users are allowed to provide disk format when creating a new Template. However this is optional. This option sets qcow2 as default disk format for every new Template created. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/kimchi/osinfo.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/kimchi/osinfo.py b/src/kimchi/osinfo.py index 3c31346..ede2541 100644 --- a/src/kimchi/osinfo.py +++ b/src/kimchi/osinfo.py @@ -33,7 +33,8 @@ SUPPORTED_ARCHS = {'x86': ('i386', 'i686', 'x86_64'), 'ppc64le': ('ppc64le')} -common_spec = {'cpus': 1, 'memory': 1024, 'disks': [{'index': 0, 'size': 10}], +common_spec = {'cpus': 1, 'memory': 1024, 'disks': [{'index': 0, 'size': 10, + 'format': 'qcow2'}], 'cdrom_bus': 'ide', 'cdrom_index': 2, 'mouse_bus': 'ps2'} -- 2.1.0

2 2

[PATCH 0/2] Objectstore and server tests
by Aline Manera 16 Apr '15

16 Apr '15

On way to complete the tests refactoring, I am sending 2 patches: one related to objectstore tests and other one about server tests. Aline Manera (2): Object store tests Server tests tests/test_mockmodel.py | 35 ------ tests/test_model.py | 71 ------------ tests/test_objectstore.py | 96 ++++++++++++++++ tests/test_rest.py | 176 ----------------------------- tests/test_server.py | 274 +++++++++++++++++++++++++++++++++++++++++++--- 5 files changed, 356 insertions(+), 296 deletions(-) create mode 100644 tests/test_objectstore.py -- 2.1.0

2 4

[PATCH] Fix disk format lock during VM creation
by Rodrigo Trujillo 16 Apr '15

16 Apr '15

Users are able to pass the disk format (qcow, raw, etc) in disk Template information. However, Kimchi is ignoring this information and always creating qcow2 disk images (if the storagepool is not 'LOGICAL') when it creates a VM based on a given Template. This patch fixes this problem, using the disk format from the template, if it does not exist, uses qcow2 as default. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/kimchi/vmtemplate.py | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/src/kimchi/vmtemplate.py b/src/kimchi/vmtemplate.py index ec477dd..de750d4 100644 --- a/src/kimchi/vmtemplate.py +++ b/src/kimchi/vmtemplate.py @@ -193,7 +193,6 @@ class VMTemplate(object): def to_volume_list(self, vm_uuid): storage_path = self._get_storage_path() - fmt = 'raw' if self._get_storage_type() in ['logical'] else 'qcow2' ret = [] for i, d in enumerate(self.info['disks']): index = d.get('index', i) @@ -201,11 +200,16 @@ class VMTemplate(object): info = {'name': volume, 'capacity': d['size'], - 'format': fmt, + 'format': d.get('format', 'qcow2'), 'path': '%s/%s' % (storage_path, volume)} + # Validate disk format + if self._get_storage_type() in ['logical'] and \ + info['format'] != 'raw': + info['format'] = 'raw' + if 'logical' == self._get_storage_type() or \ - fmt not in ['qcow2', 'raw']: + info['format'] not in ['qcow2', 'raw']: info['allocation'] = info['capacity'] else: info['allocation'] = 0 -- 2.1.0

2 2

[PATCH] issue #548: Hotplug network interfaces
by Crístian Deives 14 Apr '15

14 Apr '15

Currently, network interfaces can only be added to shutoff VMs. However, the use might want to attach an interface to a running VM as well. Allow network interfaces to be attached also while the VM is running. The related test cases have been updated to perform the same operations on both a shutoff and a running VM. Fix issue #548 (Kimchi does not support NIC hot plug) - backend only. Signed-off-by: Crístian Deives <cristiandeives(a)gmail.com> --- src/kimchi/i18n.py | 1 - src/kimchi/model/vmifaces.py | 45 ++++++++++++++--------- tests/test_model.py | 86 ++++++++++++++++++++++---------------------- 3 files changed, 73 insertions(+), 59 deletions(-) diff --git a/src/kimchi/i18n.py b/src/kimchi/i18n.py index 46990a5..d4c1cc6 100644 --- a/src/kimchi/i18n.py +++ b/src/kimchi/i18n.py @@ -118,7 +118,6 @@ messages = { "KCHVMIF0001E": _("Interface %(iface)s does not exist in virtual machine %(name)s"), "KCHVMIF0002E": _("Network %(network)s specified for virtual machine %(name)s does not exist"), - "KCHVMIF0003E": _("Do not support guest interface hot plug attachment"), "KCHVMIF0004E": _("Supported virtual machine interfaces type is only network"), "KCHVMIF0005E": _("Network name for virtual machine interface must be a string"), "KCHVMIF0006E": _("Invalid network model card specified for virtual machine interface"), diff --git a/src/kimchi/model/vmifaces.py b/src/kimchi/model/vmifaces.py index 0c10fa9..a64315d 100644 --- a/src/kimchi/model/vmifaces.py +++ b/src/kimchi/model/vmifaces.py @@ -22,7 +22,7 @@ import random import libvirt from lxml import etree, objectify -from kimchi.exception import InvalidOperation, InvalidParameter, NotFoundError +from kimchi.exception import InvalidParameter, NotFoundError from kimchi.model.config import CapabilitiesModel from kimchi.model.vms import DOM_STATE_MAP, VMModel from kimchi.xmlutils.interface import get_iface_xml @@ -48,10 +48,6 @@ class VMIfacesModel(object): raise InvalidParameter("KCHVMIF0002E", {'name': vm, 'network': params["network"]}) - dom = VMModel.get_vm(vm, self.conn) - if DOM_STATE_MAP[dom.info()[0]] != "shutoff": - raise InvalidOperation("KCHVMIF0003E") - macs = (iface.mac.get('address') for iface in self.get_vmifaces(vm, self.conn)) @@ -60,10 +56,19 @@ class VMIfacesModel(object): if params['mac'] not in macs: break + dom = VMModel.get_vm(vm, self.conn) + os_data = VMModel.vm_get_os_metadata(dom, self.caps.metadata_support) os_distro, os_version = os_data xml = get_iface_xml(params, conn.getInfo()[0], os_distro, os_version) - dom.attachDeviceFlags(xml, libvirt.VIR_DOMAIN_AFFECT_CURRENT) + + flags = 0 + if dom.isPersistent(): + flags |= libvirt.VIR_DOMAIN_AFFECT_CONFIG + if DOM_STATE_MAP[dom.info()[0]] != "shutoff": + flags |= libvirt.VIR_DOMAIN_AFFECT_LIVE + + dom.attachDeviceFlags(xml, flags) return params['mac'] @@ -118,14 +123,16 @@ class VMIfaceModel(object): dom = VMModel.get_vm(vm, self.conn) iface = self._get_vmiface(vm, mac) - if DOM_STATE_MAP[dom.info()[0]] != "shutoff": - raise InvalidOperation("KCHVMIF0003E") - if iface is None: raise NotFoundError("KCHVMIF0001E", {'name': vm, 'iface': mac}) - dom.detachDeviceFlags(etree.tostring(iface), - libvirt.VIR_DOMAIN_AFFECT_CURRENT) + flags = 0 + if dom.isPersistent(): + flags |= libvirt.VIR_DOMAIN_AFFECT_CONFIG + if DOM_STATE_MAP[dom.info()[0]] != "shutoff": + flags |= libvirt.VIR_DOMAIN_AFFECT_LIVE + + dom.detachDeviceFlags(etree.tostring(iface), flags) def update(self, vm, mac, params): dom = VMModel.get_vm(vm, self.conn) @@ -134,16 +141,22 @@ class VMIfaceModel(object): if iface is None: raise NotFoundError("KCHVMIF0001E", {'name': vm, 'iface': mac}) - # FIXME we will support to change the live VM configuration later. + flags = 0 + if dom.isPersistent(): + flags |= libvirt.VIR_DOMAIN_AFFECT_CONFIG + if DOM_STATE_MAP[dom.info()[0]] != "shutoff": + flags |= libvirt.VIR_DOMAIN_AFFECT_LIVE + if iface.attrib['type'] == 'network' and 'network' in params: iface.source.attrib['network'] = params['network'] xml = etree.tostring(iface) - dom.updateDeviceFlags(xml, flags=libvirt.VIR_DOMAIN_AFFECT_CONFIG) - # change on the persisted VM configuration only. - if 'model' in params and dom.isPersistent(): + dom.updateDeviceFlags(xml, flags=flags) + + if 'model' in params: iface.model.attrib["type"] = params['model'] xml = etree.tostring(iface) - dom.updateDeviceFlags(xml, flags=libvirt.VIR_DOMAIN_AFFECT_CONFIG) + + dom.updateDeviceFlags(xml, flags=flags) return mac diff --git a/tests/test_model.py b/tests/test_model.py index 63bd721..39e4fe6 100644 --- a/tests/test_model.py +++ b/tests/test_model.py @@ -297,9 +297,6 @@ class ModelTests(unittest.TestCase): params = {'name': 'test', 'disks': [], 'cdrom': UBUNTU_ISO} inst.templates_create(params) rollback.prependDefer(inst.template_delete, 'test') - params = {'name': 'kimchi-ifaces', 'template': '/templates/test'} - inst.vms_create(params) - rollback.prependDefer(inst.vm_delete, 'kimchi-ifaces') # Create a network net_name = 'test-network' @@ -311,45 +308,50 @@ class ModelTests(unittest.TestCase): inst.network_activate(net_name) rollback.prependDefer(inst.network_deactivate, net_name) - ifaces = inst.vmifaces_get_list('kimchi-ifaces') - self.assertEquals(1, len(ifaces)) - - iface = inst.vmiface_lookup('kimchi-ifaces', ifaces[0]) - self.assertEquals(17, len(iface['mac'])) - self.assertEquals("default", iface['network']) - self.assertIn("model", iface) - - # attach network interface to vm - iface_args = {"type": "network", - "network": "test-network", - "model": "virtio"} - mac = inst.vmifaces_create('kimchi-ifaces', iface_args) - # detach network interface from vm - rollback.prependDefer(inst.vmiface_delete, 'kimchi-ifaces', mac) - self.assertEquals(17, len(mac)) - - iface = inst.vmiface_lookup('kimchi-ifaces', mac) - self.assertEquals("network", iface["type"]) - self.assertEquals("test-network", iface['network']) - self.assertEquals("virtio", iface["model"]) - - # attach network interface to vm without providing model - iface_args = {"type": "network", - "network": "test-network"} - mac = inst.vmifaces_create('kimchi-ifaces', iface_args) - rollback.prependDefer(inst.vmiface_delete, 'kimchi-ifaces', mac) - - iface = inst.vmiface_lookup('kimchi-ifaces', mac) - self.assertEquals("network", iface["type"]) - self.assertEquals("test-network", iface['network']) - - # update vm interface - iface_args = {"network": "default", - "model": "e1000"} - inst.vmiface_update('kimchi-ifaces', mac, iface_args) - iface = inst.vmiface_lookup('kimchi-ifaces', mac) - self.assertEquals("default", iface['network']) - self.assertEquals("e1000", iface["model"]) + for vm_name in ['kimchi-ifaces', 'kimchi-ifaces-running']: + params = {'name': vm_name, 'template': '/templates/test'} + inst.vms_create(params) + rollback.prependDefer(inst.vm_delete, vm_name) + + ifaces = inst.vmifaces_get_list(vm_name) + self.assertEquals(1, len(ifaces)) + + iface = inst.vmiface_lookup(vm_name, ifaces[0]) + self.assertEquals(17, len(iface['mac'])) + self.assertEquals("default", iface['network']) + self.assertIn("model", iface) + + # attach network interface to vm + iface_args = {"type": "network", + "network": "test-network", + "model": "virtio"} + mac = inst.vmifaces_create(vm_name, iface_args) + # detach network interface from vm + rollback.prependDefer(inst.vmiface_delete, vm_name, mac) + self.assertEquals(17, len(mac)) + + iface = inst.vmiface_lookup(vm_name, mac) + self.assertEquals("network", iface["type"]) + self.assertEquals("test-network", iface['network']) + self.assertEquals("virtio", iface["model"]) + + # attach network interface to vm without providing model + iface_args = {"type": "network", + "network": "test-network"} + mac = inst.vmifaces_create(vm_name, iface_args) + rollback.prependDefer(inst.vmiface_delete, vm_name, mac) + + iface = inst.vmiface_lookup(vm_name, mac) + self.assertEquals("network", iface["type"]) + self.assertEquals("test-network", iface['network']) + + # update vm interface + iface_args = {"network": "default", + "model": "e1000"} + inst.vmiface_update(vm_name, mac, iface_args) + iface = inst.vmiface_lookup(vm_name, mac) + self.assertEquals("default", iface['network']) + self.assertEquals("e1000", iface["model"]) @unittest.skipUnless(utils.running_as_root(), 'Must be run as root') def test_vm_disk(self): -- 2.1.0

4 3

[PATCH v2 0/3] VM suspend/resume
by Crístian Deives 13 Apr '15

13 Apr '15

This is the difference between this and the previous patchset: - Use string (instead of Unicode) when comparing VM states. Crístian Viana (3): Optimize VM update function Update some VM state conditions Add support for VM suspend and resume src/kimchi/control/vms.py | 2 ++ src/kimchi/i18n.py | 4 ++++ src/kimchi/model/vms.py | 47 ++++++++++++++++++++++++++++++++++++++++++++--- tests/test_model.py | 20 ++++++++++++++++++++ tests/test_rest.py | 28 ++++++++++++++++++++++++++++ 5 files changed, 98 insertions(+), 3 deletions(-) -- 2.1.0

2 5

[PATCH 0/2] fix: Update VM stats in "lookup"
by Crístian Deives 13 Apr '15

13 Apr '15

Crístian Deives (2): Update stats when looking up one single VM Move stats-related VM functions to VMModel src/kimchi/model/vms.py | 205 +++++++++++++++++++++++------------------------- 1 file changed, 99 insertions(+), 106 deletions(-) -- 2.1.0

2 4

Re: [Kimchi-devel] Add %Used in header in Storage tab
by Aline Manera 13 Apr '15

13 Apr '15

Applied. Thanks. Regards, Aline Manera

1 0

Re: [Kimchi-devel] Add %Used in header in Storage tab
by Aline Manera 13 Apr '15

13 Apr '15

Applied. Thanks. Regards, Aline Manera

1 0