April 2015 - Kimchi-devel - oVirt List Archives

RFC - Guest memory usage
by Daniel Henrique Barboza 27 Oct '15

27 Oct '15

I was looking at the following item in the backlog: Guests Stats: Display memory utilization (use virt-df or virt-top or ...) If I understood it right, the idea here is to show the -inner- memory allocation of the guest. If you have a VM with 4Gb of RAM running an Ubuntu, we want to know how much memory the Ubuntu OS and its processes are using. I've done an investigation and I haven't found any tool to accomplish this. "virt-top", "virsh dommemstat" and the libvirt API retrieves the information of the memory usage of the guest relative to the host. In the example mentioned before, supposing that the host has 64Gb of RAM, all these tools would show that the VM is using 12% of the host RAM. They do not dive in the VM and shows the actual mem usage of the Ubuntu and its processes running there. Haven't found anything useful in other MLs and forums. The common answer is 'run top in a terminal inside the VM', which of course does not suit us. My question is: any thoughts about how we can implement this feature? Because I am starting to think that, in the end, this kind of info is strict to the guest OS and can't be polled from the outside. Thanks!

5 5

[RFC] Split Kimchi into Base Framework and Virtualization layers
by Lucio Correia 05 May '15

05 May '15

Hi, Gustavo (on cc) and me have been analyzing how to split Kimchi Virtualization layer form its Base web framework. We would like to leverage all infrastructure kimchi base framework provides (URI routing, error handling, i18n, authentication, API validation, plugin tabs loading) in other projects, and contribute this back to kimchi-ginger project. The general idea for the web framework is that it provides basic UI with login and tabs menu, and each plugin installed extends the menu with some tabs. We started by making kimchi a plugin, like ginger is. We are way advanced on that and already have most of kimchi's UI running as a plugin over an almost fully running base web framework. Ginger needs minimal changes, mainly on import statements. We can provide changes needed back to ginger too. Currently, we are splitting kimchi specific stuff like this way: (after renaming the dir src/kimchi to src/backend *) * we are using the name "backend" just for disambiguation from virtualization layer (plugins/kimchi). We will need to define a better name for that basic framework. One suggestion is to call it "wok", the base where kimchi is cooked. {src/backend => plugins/kimchi}/API.json | 0 {src/backend/control/vm => plugins/kimchi/control}/Makefile.am | 7 +- {src/backend/control/vm => plugins/kimchi/control}/__init__.py | 2 +- {src/backend => plugins/kimchi}/control/config.py | 0 {src/backend => plugins/kimchi}/control/cpuinfo.py | 0 {src/backend => plugins/kimchi}/control/debugreports.py | 0 {src/backend => plugins/kimchi}/control/groups.py | 0 {src/backend => plugins/kimchi}/control/host.py | 0 {src/backend => plugins/kimchi}/control/interfaces.py | 0 {src/backend => plugins/kimchi}/control/networks.py | 0 {src/backend => plugins/kimchi}/control/peers.py | 0 {src/backend => plugins/kimchi}/control/storagepools.py | 0 {src/backend => plugins/kimchi}/control/storageservers.py | 0 {src/backend => plugins/kimchi}/control/storagevolumes.py | 0 {src/backend => plugins/kimchi}/control/templates.py | 0 {src/backend => plugins/kimchi}/control/users.py | 0 {src/backend => plugins/kimchi}/control/vm/Makefile.am | 0 {src/backend => plugins/kimchi}/control/vm/__init__.py | 0 {src/backend => plugins/kimchi}/control/vm/hostdevs.py | 0 {src/backend => plugins/kimchi}/control/vm/ifaces.py | 0 {src/backend => plugins/kimchi}/control/vm/snapshots.py | 0 {src/backend => plugins/kimchi}/control/vm/storages.py | 0 {src/backend => plugins/kimchi}/control/vms.py | 0 {src/backend => plugins/kimchi}/disks.py | 0 {src/backend => plugins/kimchi}/imageinfo.py | 0 {src/backend => plugins/kimchi}/iscsi.py | 0 {src/backend => plugins/kimchi}/isoinfo.py | 0 {src/backend => plugins/kimchi}/kvmusertests.py | 0 {src/backend => plugins/kimchi}/mockmodel.py | 0 {src/backend/control/vm => plugins/kimchi/model}/Makefile.am | 7 +- {src/backend/control/vm => plugins/kimchi/model}/__init__.py | 8 --- {src/backend => plugins/kimchi}/model/config.py | 0 {src/backend => plugins/kimchi}/model/cpuinfo.py | 0 {src/backend => plugins/kimchi}/model/debugreports.py | 0 {src/backend => plugins/kimchi}/model/diskutils.py | 0 {src/backend => plugins/kimchi}/model/featuretests.py | 0 {src/backend => plugins/kimchi}/model/groups.py | 0 {src/backend => plugins/kimchi}/model/host.py | 0 {src/backend => plugins/kimchi}/model/hostdev.py | 0 {src/backend => plugins/kimchi}/model/interfaces.py | 0 {src/backend => plugins/kimchi}/model/libvirtconnection.py | 0 {src/backend => plugins/kimchi}/model/libvirtstoragepool.py | 0 plugins/kimchi/model/model.py | 93 ++++++++++++++++++++++++++++ {src/backend => plugins/kimchi}/model/networks.py | 0 {src/backend => plugins/kimchi}/model/peers.py | 0 {src/backend => plugins/kimchi}/model/storagepools.py | 0 {src/backend => plugins/kimchi}/model/storageservers.py | 0 {src/backend => plugins/kimchi}/model/storagetargets.py | 0 {src/backend => plugins/kimchi}/model/storagevolumes.py | 0 {src/backend => plugins/kimchi}/model/templates.py | 0 {src/backend => plugins/kimchi}/model/users.py | 0 {src/backend => plugins/kimchi}/model/utils.py | 0 {src/backend => plugins/kimchi}/model/vmhostdevs.py | 0 {src/backend => plugins/kimchi}/model/vmifaces.py | 0 {src/backend => plugins/kimchi}/model/vms.py | 0 {src/backend => plugins/kimchi}/model/vmsnapshots.py | 0 {src/backend => plugins/kimchi}/model/vmstorages.py | 0 {src/backend => plugins/kimchi}/osinfo.py | 0 {src/backend => plugins/kimchi}/scan.py | 0 {config => plugins/kimchi/ui}/Makefile.am | 0 {config/ui => plugins/kimchi/ui/config}/Makefile.am | 0 config/ui/tabs.xml => plugins/kimchi/ui/config/tab-ext.xml | 0 {src/backend => plugins/kimchi}/vmtemplate.py | 0 The remaining generic stuff on the "backend" side, usable or already in use by other plugins, would be: backend |___model | |___model.py | |___plugins.py | |___tasks.py |___control | |___base.py | |___plugins.py | |___tasks.py | |___utils.py |___xmlutils | |___network.py | |___disk.py | |___utils.py | |___graphics.py | |___qemucmdline.py | |___interface.py |___asynctask.py |___auth.py |___basemodel.py |___cachebust.py |___distroloader.py |___exception.py |___i18n.py |___netinfo.py |___network.py |___objectstore.py |___proxy.py |___repositories.py |___rollbackcontext.py |___root.py |___screenshot.py |___server.py |___sslcert.py |___swupdate.py |___template.py |___utils.py |___vnc.py We don't want to make this a too long e-mail and lose the focus on the idea. We have further technical details to discuss in a follow up. Please let us know your thoughts on that, any directions and requirements you have on this, and how could we proceed with that plan. -- Lucio Correia Software Engineer IBM LTC Brazil

2 3

[kimchi-devel][PATCHv4 0/7] Upload storage volume
by lvroyce＠linux.vnet.ibm.com 04 May '15

04 May '15

From: Royce Lv <lvroyce(a)linux.vnet.ibm.com> How to upload: POST /storagepools/<pool-name>/storagevolumes/ {'capacity':size-of-the-volume,'name':name-of-the-volume,'format':'raw'} PUT /storagepools/<pool-name>/storagevolumes/<volume-name> {'chunk':chunk-data,'chunk_size':size-of-chunk,'index':index-of-the-chunk-of-uploaded-file} PATCH 7/7 is a script against real model server, you can use it to verify the logic of upload. Royce Lv (7): Update docs and json schema of storage volume upload Update controller to make update accept formdata params Add lock facility for storage volume upload Update model for storage volume update Fix incomplete record when uploading update test case for storage volume upload A test script against real model server docs/API.md | 7 ++- src/kimchi/API.json | 22 ++++++++ src/kimchi/control/base.py | 6 +-- src/kimchi/i18n.py | 4 ++ src/kimchi/isoinfo.py | 5 +- src/kimchi/model/storagevolumes.py | 61 ++++++++++----------- src/kimchi/model/utils.py | 46 ++++++++++++++++ tests/test_api.py | 108 +++++++++++++++++++++++++++++++++++++ tests/test_model_storagevolume.py | 95 ++++++++++++++++++++++++-------- 9 files changed, 291 insertions(+), 63 deletions(-) create mode 100644 tests/test_api.py -- 2.1.0

3 15

[PATCH 0/5] Support vm name with special characters
by Ramon Medeiros 04 May '15

04 May '15

Ramon Medeiros (5): Create title tag Change the way that get_vm() and vm_list() search for VMs Use get_vm() instead of conn.lookupByName Encode vm_name to generate VNC token Reflect the unicode conversion on token creation src/kimchi/model/vms.py | 50 +++++++++++++++++++++++++++++++++--------- src/kimchi/model/vmstorages.py | 4 ++-- src/kimchi/vmtemplate.py | 4 +++- ui/js/src/kimchi.api.js | 4 ++-- 4 files changed, 47 insertions(+), 15 deletions(-) -- 2.1.0

2 11

[PATCH v2] Supress error messages while checking vm metadata
by Ramon Medeiros 04 May '15

04 May '15

Changes v2: Use parse functions present on model/utils.py The function FeatureTests.disable_libvirt_error_logging only removes the error message that will be displayed on the console. Libvirt also logs errors on messages log. To avoid flood messages log with metadata errors, it will be verified manually if the xml has the tag. Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> --- src/kimchi/model/utils.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/kimchi/model/utils.py b/src/kimchi/model/utils.py index b2739b2..1b114f6 100644 --- a/src/kimchi/model/utils.py +++ b/src/kimchi/model/utils.py @@ -95,6 +95,9 @@ def _kimchi_set_metadata_node(dom, node): def libvirt_get_kimchi_metadata_node(dom, mode="current"): + if _kimchi_get_metadata_node(dom, "metadata") == None: + return + FeatureTests.disable_libvirt_error_logging() try: xml = dom.metadata(libvirt.VIR_DOMAIN_METADATA_ELEMENT, -- 2.1.0

2 1

[PATCH 0/3] Snapshot enhancements
by Crístian Deives 04 May '15

04 May '15

Crístian Deives (3): snapshot: Handle non-existing snapshots in mock lookup snapshot: Change the action 'revert' to a Task snapshot: Allow creating a snapshot of a domain in any state src/kimchi/control/vm/snapshots.py | 2 +- src/kimchi/i18n.py | 1 - src/kimchi/mockmodel.py | 14 ++++++++++++++ src/kimchi/model/vmsnapshots.py | 36 ++++++++++++++++++++++-------------- tests/test_model.py | 15 +++++++++++---- tests/test_rest.py | 13 +++++++++++++ 6 files changed, 61 insertions(+), 20 deletions(-) -- 2.1.0

3 9

[PATCH] Do not resolve hostname to IP in using remote ISOs
by Aline Manera 04 May '15

04 May '15

Due a limitation in old versions of qemu-io (qemu-io was not able to resolve hostnames preveting the ISO streaming support), Kimchi automatically resolved the hostname to IP to workaround this problem. As all the most used Linux distributions (RHEl7.1, Ubuntu 14.10, Fedora 21 and openSUSE 13.2) have a working version of qemu-io, Kimchi does not need to care about it anymore. It also fixes the issue #436 Signed-off-by: Aline Manera <alinefm(a)linux.vnet.ibm.com> Signed-off-by: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com> --- src/kimchi/model/config.py | 2 -- src/kimchi/model/featuretests.py | 26 -------------------------- src/kimchi/model/utils.py | 11 ----------- src/kimchi/model/vms.py | 1 - src/kimchi/model/vmstorages.py | 10 +++------- src/kimchi/vmtemplate.py | 12 ++---------- tests/test_model.py | 11 +---------- 7 files changed, 6 insertions(+), 67 deletions(-) diff --git a/src/kimchi/model/config.py b/src/kimchi/model/config.py index a417c9b..ca0a74b 100644 --- a/src/kimchi/model/config.py +++ b/src/kimchi/model/config.py @@ -51,7 +51,6 @@ class CapabilitiesModel(object): def __init__(self, **kargs): self.conn = kargs['conn'] self.qemu_stream = False - self.qemu_stream_dns = False self.libvirt_stream_protocols = [] self.fc_host_support = False self.metadata_support = False @@ -88,7 +87,6 @@ class CapabilitiesModel(object): kimchi_log.info("*** Running feature tests ***") conn = self.conn.get() self.qemu_stream = FeatureTests.qemu_supports_iso_stream() - self.qemu_stream_dns = FeatureTests.qemu_iso_stream_dns() self.nfs_target_probe = FeatureTests.libvirt_support_nfs_probe(conn) self.fc_host_support = FeatureTests.libvirt_support_fc_host(conn) self.metadata_support = FeatureTests.has_metadata_support(conn) diff --git a/src/kimchi/model/featuretests.py b/src/kimchi/model/featuretests.py index 8cbe31d..9400151 100644 --- a/src/kimchi/model/featuretests.py +++ b/src/kimchi/model/featuretests.py @@ -21,9 +21,7 @@ import cherrypy import libvirt import lxml.etree as ET import platform -import socket import subprocess -import threading from lxml.builder import E @@ -153,30 +151,6 @@ class FeatureTests(object): return len(stderr) == 0 @staticmethod - @servermethod - def qemu_iso_stream_dns(): - host = socket.getfqdn(cherrypy.server.socket_host) - port = cherrypy.server.socket_port - cmd = ["qemu-io", "-r", "http://%s:%d/images/icon-fedora.png" % - (host, port), "-c", "'read -v 0 512'"] - proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - - thread = threading.Thread(target=proc.communicate) - thread.start() - thread.join(5) - - if thread.is_alive(): - proc.kill() - thread.join() - return False - - if proc.returncode != 0: - return False - - return True - - @staticmethod def libvirt_support_fc_host(conn): try: FeatureTests.disable_libvirt_error_logging() diff --git a/src/kimchi/model/utils.py b/src/kimchi/model/utils.py index b2739b2..0c9d519 100644 --- a/src/kimchi/model/utils.py +++ b/src/kimchi/model/utils.py @@ -18,8 +18,6 @@ # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA import libvirt -import socket -import urlparse from lxml import etree, objectify from lxml.builder import E, ElementMaker @@ -41,15 +39,6 @@ def get_vm_name(vm_name, t_name, name_list): raise OperationFailed("KCHUTILS0003E") -def check_remote_disk_path(path, qemu_stream_dns): - hostname = urlparse.urlparse(path).hostname - if hostname is not None and not qemu_stream_dns: - ip = socket.gethostbyname(hostname) - return path.replace(hostname, ip) - - return path - - def get_vm_config_flag(dom, mode="persistent"): # libvirt.VIR_DOMAIN_AFFECT_CURRENT is 0 # VIR_DOMAIN_AFFECT_LIVE is 1, VIR_DOMAIN_AFFECT_CONFIG is 2 diff --git a/src/kimchi/model/vms.py b/src/kimchi/model/vms.py index c8267c1..a6ca27b 100644 --- a/src/kimchi/model/vms.py +++ b/src/kimchi/model/vms.py @@ -125,7 +125,6 @@ class VMsModel(object): stream_protocols = self.caps.libvirt_stream_protocols xml = t.to_vm_xml(name, vm_uuid, libvirt_stream_protocols=stream_protocols, - qemu_stream_dns=self.caps.qemu_stream_dns, graphics=graphics, volumes=vol_list) diff --git a/src/kimchi/model/vmstorages.py b/src/kimchi/model/vmstorages.py index 37aca64..87c6b3d 100644 --- a/src/kimchi/model/vmstorages.py +++ b/src/kimchi/model/vmstorages.py @@ -1,7 +1,7 @@ # # Project Kimchi # -# Copyright IBM, Corp. 2014 +# Copyright IBM, Corp. 2014-2015 # # This library is free software; you can redistribute it and/or # modify it under the terms of the GNU Lesser General Public @@ -26,7 +26,7 @@ from kimchi.exception import OperationFailed from kimchi.model.config import CapabilitiesModel from kimchi.model.vms import DOM_STATE_MAP, VMModel from kimchi.model.storagevolumes import StorageVolumeModel -from kimchi.model.utils import check_remote_disk_path, get_vm_config_flag +from kimchi.model.utils import get_vm_config_flag from kimchi.osinfo import lookup from kimchi.model.diskutils import get_disk_ref_cnt, set_disk_ref_cnt from kimchi.utils import kimchi_log @@ -131,8 +131,6 @@ class VMStoragesModel(object): params['disk'] = vol_info['type'] params.update(self._get_available_bus_address(params['bus'], vm_name)) - params['path'] = check_remote_disk_path(params['path'], - self.caps.qemu_stream_dns) # Add device to VM dev, xml = get_disk_xml(params) @@ -215,9 +213,7 @@ class VMStorageModel(object): if dev_info['type'] != 'cdrom': raise InvalidOperation("KCHVMSTOR0006E") - params['path'] = check_remote_disk_path(params.get('path', ''), - self.caps.qemu_stream_dns) - + params['path'] = params.get('path', '') old_disk_path = dev_info['path'] new_disk_path = params['path'] if new_disk_path != old_disk_path: diff --git a/src/kimchi/vmtemplate.py b/src/kimchi/vmtemplate.py index ef97d83..e047228 100644 --- a/src/kimchi/vmtemplate.py +++ b/src/kimchi/vmtemplate.py @@ -18,7 +18,6 @@ # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA import os -import socket import stat import time import urlparse @@ -128,7 +127,7 @@ class VMTemplate(object): except IsoFormatError: raise InvalidParameter("KCHISO0001E", {'filename': iso}) - def _get_cdrom_xml(self, libvirt_stream_protocols, qemu_stream_dns): + def _get_cdrom_xml(self, libvirt_stream_protocols): if 'cdrom' not in self.info: return '' @@ -139,11 +138,6 @@ class VMTemplate(object): params['index'] = self.info['cdrom_index'] params['path'] = self.info['cdrom'] - hostname = urlparse.urlparse(params['path']).hostname - if hostname is not None and not qemu_stream_dns: - ip = socket.gethostbyname(hostname) - params['path'] = params['path'].replace(hostname, ip) - if self.info.get('iso_stream', False): protocol = urlparse.urlparse(params['path']).scheme if protocol not in libvirt_stream_protocols: @@ -304,10 +298,8 @@ class VMTemplate(object): graphics.update(kwargs.get('graphics', {})) params['graphics'] = get_graphics_xml(graphics) - qemu_stream_dns = kwargs.get('qemu_stream_dns', False) libvirt_stream_protocols = kwargs.get('libvirt_stream_protocols', []) - cdrom_xml = self._get_cdrom_xml(libvirt_stream_protocols, - qemu_stream_dns) + cdrom_xml = self._get_cdrom_xml(libvirt_stream_protocols) if not urlparse.urlparse(self.info.get('cdrom', "")).scheme in \ libvirt_stream_protocols and \ diff --git a/tests/test_model.py b/tests/test_model.py index 210adfd..bd195b5 100644 --- a/tests/test_model.py +++ b/tests/test_model.py @@ -23,9 +23,7 @@ import os import pwd import re import shutil -import socket import time -import urlparse import unittest import uuid @@ -564,14 +562,7 @@ class ModelTests(unittest.TestCase): {'path': valid_remote_iso_path}) cdrom_info = inst.vmstorage_lookup(vm_name, cdrom_dev) cur_cdrom_path = re.sub(":80/", '/', cdrom_info['path']) - - # As Kimchi server is not running during this test case - # CapabilitiesModel.qemu_stream_dns will be always False - # so we need to convert the hostname to IP - output = urlparse.urlparse(valid_remote_iso_path) - hostname = socket.gethostbyname(output.hostname) - url = valid_remote_iso_path.replace(output.hostname, hostname) - self.assertEquals(url, cur_cdrom_path) + self.assertEquals(valid_remote_iso_path, cur_cdrom_path) @unittest.skipUnless(utils.running_as_root(), 'Must be run as root') def test_vm_storage_provisioning(self): -- 2.1.0

2 2

[PATCH 0/7] Make Template defaults configurable
by Aline Manera 30 Apr '15

30 Apr '15

Aline Manera (7): Remove useless variable in osinfo.py Merge common_spec wiht defaults value in osinfo.py Make Template defaults configurable Add libvirt-daemon-config-network package as Kimchi dependency Verify all networks set as Template defaults prior to server start up Create option to auto create ISO pool or not on server start up Verify storage pool set as Template default prior to server starts up contrib/DEBIAN/control.in | 1 + contrib/kimchi.spec.fedora.in | 3 ++ contrib/kimchi.spec.suse.in | 3 ++ docs/README.md | 14 +++--- src/Makefile.am | 2 +- src/kimchi.conf.in | 3 ++ src/kimchi/config.py.in | 1 + src/kimchi/model/model.py | 44 +---------------- src/kimchi/model/networks.py | 61 ++++++++++------------- src/kimchi/model/storagepools.py | 66 +++++++++++++++++++++++++ src/kimchi/model/templates.py | 2 - src/kimchi/osinfo.py | 101 ++++++++++++++++++++++++++++++--------- src/kimchid.in | 3 ++ src/template.conf | 47 ++++++++++++++++++ 14 files changed, 240 insertions(+), 111 deletions(-) create mode 100644 src/template.conf -- 2.1.0

2 15

[PATCH 0/6 v7] Asyn VM creation
by Aline Manera 30 Apr '15

30 Apr '15

V6 -> V7: - Update test cases - Update UI to reuse the clone box for pending VM creation Aline Manera (1): Create VMs Asynchronously: UI Christy Perez (5): Append clone to target_uri for vm clone task Tests for new clone target_uri UI changes for new clone target_uri Create VMs asynchronously: Backend Create VMs Asynchronously: Tests src/kimchi/control/vms.py | 4 +-- src/kimchi/model/vms.py | 33 ++++++++++++++++++----- tests/test_authorization.py | 25 +++++++++++------- tests/test_mockmodel.py | 14 +++++++--- tests/test_model.py | 49 +++++++++++++++++++++++----------- tests/test_rest.py | 60 +++++++++++++++++++++++++++++++----------- tests/test_server.py | 26 ++++++++---------- ui/css/theme-default/list.css | 8 +++--- ui/js/src/kimchi.guest_main.js | 32 ++++++++++++++++++---- ui/pages/guest.html.tmpl | 4 +-- ui/pages/i18n.json.tmpl | 1 + 11 files changed, 176 insertions(+), 80 deletions(-) -- 2.1.0

2 9

kimchi Chef cookbook
by Scott Hain 30 Apr '15

30 Apr '15

Hello, I wanted to share that I've put together a Chef cookbook for running a local dev/test instance of kimchi. I am using it for my own testing and I thought it might be useful to someone else. https://github.com/opscode-cookbooks/kimchi I created for internal testing, but it could easily be modified to set up hardware or other VMs (it may work already for these use cases, but I haven't tested them). Thanks! -- *Scott Hain* — Software Engineer II 360.360.7433 – shain(a)chef.io – *my: *Linkedin <http://www.linkedin.com/> Twitter <http://www.twitter.com/elejia> *CHEF* CHEF.IO <http://chef.io> TM *chef.io* <http://www.chef.io/> Blog <http://www.chef.io/blog/> Facebook <https://www.facebook.com/getchefdotcom> Twitter <https://twitter.com/chef> Youtube <https://www.youtube.com/getchef>

2 1

[PATCH 5/6] Create VMs Asynchronously: Tests
by Christy Perez 28 Apr '15

28 Apr '15

Signed-off-by: Christy Perez <christy(a)linux.vnet.ibm.com> --- tests/test_authorization.py | 23 +++++++++++------- tests/test_mockmodel.py | 12 +++++++--- tests/test_model.py | 46 +++++++++++++++++++++++++---------- tests/test_rest.py | 58 ++++++++++++++++++++++++++++++++++----------- 4 files changed, 100 insertions(+), 39 deletions(-) diff --git a/tests/test_authorization.py b/tests/test_authorization.py index 4fcc496..eae837c 100644 --- a/tests/test_authorization.py +++ b/tests/test_authorization.py @@ -26,7 +26,7 @@ import kimchi.mockmodel from iso_gen import construct_fake_iso from utils import get_free_port, patch_auth, request -from utils import run_server +from utils import run_server, wait_task test_server = None @@ -118,19 +118,24 @@ def test_nonroot_access(self): # Non-root users can only get vms authorized to them model.templates_create({'name': u'test', 'cdrom': fake_iso}) - model.vms_create({'name': u'test-me', 'template': '/templates/test'}) + task_info = model.vms_create({'name': u'test-me', + 'template': '/templates/test'}) + wait_task(model.task_lookup, task_info['id']) + model.vm_update(u'test-me', {'users': [kimchi.mockmodel.fake_user.keys()[0]], 'groups': []}) - model.vms_create({'name': u'test-usera', - 'template': '/templates/test'}) + task_info = model.vms_create({'name': u'test-usera', + 'template': '/templates/test'}) + wait_task(model.task_lookup, task_info['id']) non_root = list(set(model.users_get_list()) - set(['root']))[0] model.vm_update(u'test-usera', {'users': [non_root], 'groups': []}) - model.vms_create({'name': u'test-groupa', - 'template': '/templates/test'}) + task_info = model.vms_create({'name': u'test-groupa', + 'template': '/templates/test'}) + wait_task(model.task_lookup, task_info['id']) a_group = model.groups_get_list()[0] model.vm_update(u'test-groupa', {'groups': [a_group]}) @@ -143,9 +148,9 @@ def test_nonroot_access(self): self.assertEquals(403, resp.status) # Create a vm using mockmodel directly to test Resource access - model.vms_create({'name': 'kimchi-test', - 'template': '/templates/test'}) - + task_info = model.vms_create({'name': 'kimchi-test', + 'template': '/templates/test'}) + wait_task(model.task_lookup, task_info['id']) resp = self.request('/vms/kimchi-test', '{}', 'PUT') self.assertEquals(403, resp.status) resp = self.request('/vms/kimchi-test', '{}', 'DELETE') diff --git a/tests/test_mockmodel.py b/tests/test_mockmodel.py index aa48dd1..a731579 100644 --- a/tests/test_mockmodel.py +++ b/tests/test_mockmodel.py @@ -66,7 +66,9 @@ def test_screenshot_refresh(self): req = json.dumps({'name': 'test', 'cdrom': fake_iso}) request(host, ssl_port, '/templates', req, 'POST') req = json.dumps({'name': 'test-vm', 'template': '/templates/test'}) - request(host, ssl_port, '/vms', req, 'POST') + resp = request(host, ssl_port, '/vms', req, 'POST') + task = json.loads(resp.read()) + wait_task(model.task_lookup, task['id']) # Test screenshot refresh for running vm request(host, ssl_port, '/vms/test-vm/start', '{}', 'POST') @@ -94,7 +96,9 @@ def test_vm_list_sorted(self): def add_vm(name): # Create a VM req = json.dumps({'name': name, 'template': '/templates/test'}) - request(host, ssl_port, '/vms', req, 'POST') + task = json.loads(request(host, ssl_port, '/vms', req, + 'POST').read()) + wait_task(model.task_lookup, task['id']) vms = [u'abc', u'bca', u'cab', u'xba'] for vm in vms: @@ -106,7 +110,9 @@ def add_vm(name): def test_vm_info(self): model.templates_create({'name': u'test', 'cdrom': fake_iso}) - model.vms_create({'name': u'test-vm', 'template': '/templates/test'}) + task = model.vms_create({'name': u'test-vm', + 'template': '/templates/test'}) + wait_task(model.task_lookup, task['id']) vms = model.vms_get_list() self.assertEquals(2, len(vms)) self.assertIn(u'test-vm', vms) diff --git a/tests/test_model.py b/tests/test_model.py index ad0dccd..7e6090e 100644 --- a/tests/test_model.py +++ b/tests/test_model.py @@ -129,8 +129,11 @@ def test_vm_lifecycle(self): rollback.prependDefer(inst.template_delete, 'test') params = {'name': 'kimchi-vm', 'template': '/templates/test'} - inst.vms_create(params) + task = inst.vms_create(params) rollback.prependDefer(inst.vm_delete, 'kimchi-vm') + inst.task_wait(task['id'], 10) + task = inst.task_lookup(task['id']) + self.assertEquals('finished', task['status']) vms = inst.vms_get_list() self.assertTrue('kimchi-vm' in vms) @@ -267,7 +270,8 @@ def test_image_based_template(self): session.store('template', tmpl_name, tmpl_info) params = {'name': 'kimchi-vm', 'template': '/templates/img-tmpl'} - inst.vms_create(params) + task = inst.vms_create(params) + inst.task_wait(task['id']) rollback.prependDefer(inst.vm_delete, 'kimchi-vm') vms = inst.vms_get_list() @@ -286,7 +290,8 @@ def test_vm_graphics(self): inst.templates_create(params) with RollbackContext() as rollback: params = {'name': 'kimchi-vnc', 'template': '/templates/test'} - inst.vms_create(params) + task1 = inst.vms_create(params) + inst.task_wait(task1['id']) rollback.prependDefer(inst.vm_delete, 'kimchi-vnc') info = inst.vm_lookup('kimchi-vnc') @@ -296,7 +301,8 @@ def test_vm_graphics(self): graphics = {'type': 'spice', 'listen': '127.0.0.1'} params = {'name': 'kimchi-spice', 'template': '/templates/test', 'graphics': graphics} - inst.vms_create(params) + task2 = inst.vms_create(params) + inst.task_wait(task2['id']) rollback.prependDefer(inst.vm_delete, 'kimchi-spice') info = inst.vm_lookup('kimchi-spice') @@ -312,6 +318,10 @@ def test_vm_ifaces(self): params = {'name': 'test', 'disks': [], 'cdrom': UBUNTU_ISO} inst.templates_create(params) rollback.prependDefer(inst.template_delete, 'test') + params = {'name': 'kimchi-ifaces', 'template': '/templates/test'} + task = inst.vms_create(params) + inst.task_wait(task['id']) + rollback.prependDefer(inst.vm_delete, 'kimchi-ifaces') # Create a network net_name = 'test-network' @@ -422,7 +432,8 @@ def _attach_disk(expect_bus=modern_disk_bus): inst.templates_create(params) rollback.prependDefer(inst.template_delete, 'test') params = {'name': vm_name, 'template': '/templates/test'} - inst.vms_create(params) + task1 = inst.vms_create(params) + inst.task_wait(task1['id']) rollback.prependDefer(inst.vm_delete, vm_name) prev_count = len(inst.vmstorages_get_list(vm_name)) @@ -465,7 +476,8 @@ def _attach_disk(expect_bus=modern_disk_bus): rollback.prependDefer(inst.template_delete, 'old_distro_template') params = {'name': vm_name, 'template': '/templates/old_distro_template'} - inst.vms_create(params) + task2 = inst.vms_create(params) + inst.task_wait(task2['id']) rollback.prependDefer(inst.vm_delete, vm_name) # Need to check the right disk_bus for old distro @@ -486,7 +498,8 @@ def test_vm_cdrom(self): inst.templates_create(params) rollback.prependDefer(inst.template_delete, 'test') params = {'name': vm_name, 'template': '/templates/test'} - inst.vms_create(params) + task = inst.vms_create(params) + inst.task_wait(task['id']) rollback.prependDefer(inst.vm_delete, vm_name) prev_count = len(inst.vmstorages_get_list(vm_name)) @@ -575,7 +588,8 @@ def test_vm_storage_provisioning(self): rollback.prependDefer(inst.template_delete, 'test') params = {'name': 'test-vm-1', 'template': '/templates/test'} - inst.vms_create(params) + task = inst.vms_create(params) + inst.task_wait(task['id']) rollback.prependDefer(inst.vm_delete, 'test-vm-1') vm_info = inst.vm_lookup(params['name']) @@ -596,10 +610,12 @@ def test_vm_edit(self): with RollbackContext() as rollback: params_1 = {'name': 'kimchi-vm1', 'template': '/templates/test'} params_2 = {'name': 'kimchi-vm2', 'template': '/templates/test'} - inst.vms_create(params_1) + task1 = inst.vms_create(params_1) + inst.task_wait(task1['id']) rollback.prependDefer(utils.rollback_wrapper, inst.vm_delete, 'kimchi-vm1') - inst.vms_create(params_2) + task2 = inst.vms_create(params_2) + inst.task_wait(task2['id']) rollback.prependDefer(utils.rollback_wrapper, inst.vm_delete, 'kimchi-vm2') @@ -778,11 +794,13 @@ def test_delete_running_vm(self): rollback.prependDefer(inst.template_delete, 'test') params = {'name': u'kīмсhī-∨м', 'template': u'/templates/test'} - inst.vms_create(params) + task = inst.vms_create(params) + inst.task_wait(task['id']) rollback.prependDefer(utils.rollback_wrapper, inst.vm_delete, u'kīмсhī-∨м') inst.vm_start(u'kīмсhī-∨м') + self.assertEquals(inst.vm_lookup(u'kīмсhī-∨м')['state'], 'running') rollback.prependDefer(utils.rollback_wrapper, inst.vm_poweroff, u'kīмсhī-∨м') @@ -801,7 +819,8 @@ def test_vm_list_sorted(self): rollback.prependDefer(inst.template_delete, 'test') params = {'name': 'kimchi-vm', 'template': '/templates/test'} - inst.vms_create(params) + task = inst.vms_create(params) + inst.task_wait(task['id']) rollback.prependDefer(inst.vm_delete, 'kimchi-vm') vms = inst.vms_get_list() @@ -873,7 +892,8 @@ def test_use_test_host(self): params = {'name': 'kimchi-vm', 'template': '/templates/test'} - inst.vms_create(params) + task = inst.vms_create(params) + inst.task_wait(task['id']) rollback.prependDefer(inst.vm_delete, 'kimchi-vm') vms = inst.vms_get_list() diff --git a/tests/test_rest.py b/tests/test_rest.py index 65c3db5..8826424 100644 --- a/tests/test_rest.py +++ b/tests/test_rest.py @@ -112,7 +112,9 @@ def test_get_vms(self): req = json.dumps({'name': name, 'template': '/templates/test', 'users': test_users, 'groups': test_groups}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) vms = json.loads(self.request('/vms').read()) self.assertEquals(11, len(vms)) @@ -130,7 +132,9 @@ def test_edit_vm(self): req = json.dumps({'name': 'vm-1', 'template': '/templates/test'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) vm = json.loads(self.request('/vms/vm-1').read()) self.assertEquals('vm-1', vm['name']) @@ -247,7 +251,9 @@ def test_vm_lifecycle(self): # Create a VM req = json.dumps({'name': 'test-vm', 'template': '/templates/test'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) + self.assertEquals(202, resp.status) # Verify the VM vm = json.loads(self.request('/vms/test-vm').read()) @@ -429,7 +435,9 @@ def test_vm_graphics(self): # Create a VM with default args req = json.dumps({'name': 'test-vm', 'template': '/templates/test'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Verify the VM vm = json.loads(self.request('/vms/test-vm').read()) self.assertEquals('127.0.0.1', vm['graphics']['listen']) @@ -443,7 +451,9 @@ def test_vm_graphics(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test', 'graphics': graphics}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Verify the VM vm = json.loads(self.request('/vms/test-vm').read()) self.assertEquals('127.0.0.1', vm['graphics']['listen']) @@ -457,7 +467,9 @@ def test_vm_graphics(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test', 'graphics': graphics}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Verify the VM vm = json.loads(self.request('/vms/test-vm').read()) self.assertEquals('fe00::0', vm['graphics']['listen']) @@ -471,7 +483,9 @@ def test_vm_graphics(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test', 'graphics': graphics}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Verify the VM vm = json.loads(self.request('/vms/test-vm').read()) self.assertEquals('127.0.0.1', vm['graphics']['listen']) @@ -513,7 +527,9 @@ def test_vm_storage_devices(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Delete the VM rollback.prependDefer(self.request, '/vms/test-vm', '{}', 'DELETE') @@ -658,7 +674,9 @@ def test_vm_iface(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Delete the VM rollback.prependDefer(self.request, '/vms/test-vm', '{}', 'DELETE') @@ -738,7 +756,10 @@ def test_vm_customise_storage(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test', 'storagepool': '/storagepools/alt'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) + resp = self.request('/vms/test-vm', {}, 'GET') vm_info = json.loads(resp.read()) # Test template not changed after vm customise its pool @@ -791,7 +812,9 @@ def test_scsi_fc_storage(self): req = json.dumps({'name': 'test-vm', 'template': '/templates/test_fc_pool'}) resp = self.request('/vms', req, 'POST') - self.assertEquals(201, resp.status) + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Start the VM resp = self.request('/vms/test-vm/start', '{}', 'POST') @@ -816,8 +839,10 @@ def test_unnamed_vms(self): # Create 5 unnamed vms from this template for i in xrange(1, 6): req = json.dumps({'template': '/templates/test'}) - vm = json.loads(self.request('/vms', req, 'POST').read()) - self.assertEquals('test-vm-%i' % i, vm['name']) + task = json.loads(self.request('/vms', req, 'POST').read()) + wait_task(self._task_lookup, task['id']) + resp = self.request('/vms/test-vm-%i' % i, {}, 'GET') + self.assertEquals(resp.status, 200) count = len(json.loads(self.request('/vms').read())) self.assertEquals(6, count) @@ -849,7 +874,10 @@ def test_create_vm_with_img_based_template(self): self.assertEquals(201, resp.status) req = json.dumps({'template': '/templates/test'}) - json.loads(self.request('/vms', req, 'POST').read()) + resp = self.request('/vms', req, 'POST') + self.assertEquals(202, resp.status) + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Test storage volume created with backing store of base file resp = json.loads( @@ -939,6 +967,8 @@ def test_screenshot_refresh(self): resp = self.request('/templates', req, 'POST') req = json.dumps({'name': 'test-vm', 'template': '/templates/test'}) resp = self.request('/vms', req, 'POST') + task = json.loads(resp.read()) + wait_task(self._task_lookup, task['id']) # Test screenshot for shut-off state vm resp = self.request('/vms/test-vm/screenshot') -- 2.1.0

1 0

[PATCH 1/6] Append clone to target_uri for vm clone task
by Christy Perez 28 Apr '15

28 Apr '15

So that we can differentiate between clone and create tasks for VMs. Signed-off-by: Christy Perez <christy(a)linux.vnet.ibm.com> --- src/kimchi/model/vms.py | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/src/kimchi/model/vms.py b/src/kimchi/model/vms.py index a6ca27b..5f8c3da 100644 --- a/src/kimchi/model/vms.py +++ b/src/kimchi/model/vms.py @@ -229,9 +229,8 @@ def clone(self, name): new_name = get_next_clone_name(current_vm_names, name) # create a task with the actual clone function - taskid = add_task(u'/vms/%s' % new_name, self._clone_task, - self.objstore, - {'name': name, 'new_name': new_name}) + taskid = add_task(u'/vms/%s/clone' % new_name, self._clone_task, + self.objstore, {'name': name, 'new_name': new_name}) return self.task.lookup(taskid) -- 2.1.0

1 1

[PATCH v6 0/6] Async VM Creation
by Christy Perez 28 Apr '15

28 Apr '15

v6 Changes: - Rebase v5 Changes: - Rebase - Remove all the unimperative target_uri changes (see below) If a guest has a large disk, and uses a filesystem that requires preallocation, it can take several minutes to create a VM. During that time, kimchi is tied up by the VM creation. This patch changes the VMs Collection to be an AsyncCollection. Another change required for this was to create a more granular way to query vm-related tasks. The original idea was to add another field to the task database, but then Aline suggested just modifying the task_uri. Since the task_uri cretation will be changed in a future patchset, this is now only modified for the conflicting (clone) tasks. Christy Perez (6): Append clone to target_uri for vm clone task Tests for new clone target_uri UI changes for new clone target_uri Create VMs asynchronously: Backend Create VMs Asynchronously: Tests Create VMs Asynchronously: UI src/kimchi/control/vms.py | 4 +-- src/kimchi/model/vms.py | 32 +++++++++++++++++---- tests/test_authorization.py | 23 +++++++++------ tests/test_mockmodel.py | 12 ++++++-- tests/test_model.py | 50 ++++++++++++++++++++++---------- tests/test_model_storagevolume.py | 2 +- tests/test_rest.py | 60 +++++++++++++++++++++++++++++---------- ui/css/theme-default/list.css | 18 ++++++++++++ ui/js/src/kimchi.guest_main.js | 29 +++++++++++++++---- ui/pages/guest.html.tmpl | 3 ++ 10 files changed, 177 insertions(+), 56 deletions(-) -- 2.1.0

1 5

[PATCH 6/6] Create VMs Asynchronously: UI
by Christy Perez 28 Apr '15

28 Apr '15

Signed-off-by: Christy Perez <christy(a)linux.vnet.ibm.com> --- ui/css/theme-default/list.css | 18 ++++++++++++++++++ ui/js/src/kimchi.guest_main.js | 25 ++++++++++++++++++++++--- ui/pages/guest.html.tmpl | 3 +++ 3 files changed, 43 insertions(+), 3 deletions(-) diff --git a/ui/css/theme-default/list.css b/ui/css/theme-default/list.css index 7b32ea6..62d1539 100644 --- a/ui/css/theme-default/list.css +++ b/ui/css/theme-default/list.css @@ -306,3 +306,21 @@ margin-left: 5px; text-shadow: -1px -1px 1px #CCCCCC, 1px 1px 1px #FFFFFF; } + +.guest-create { + margin: 10px; +} + +.guest-create .icon { + background: url('../../images/theme-default/kimchi-loading15x15.gif') no-repeat; + display: inline-block; + width: 20px; + height: 20px; + vertical-align: middle; +} + +.guest-create .text { + color: #666666; + margin-left: 5px; + text-shadow: -1px -1px 1px #CCCCCC, 1px 1px 1px #FFFFFF; +} diff --git a/ui/js/src/kimchi.guest_main.js b/ui/js/src/kimchi.guest_main.js index c712bb4..dbcc162 100644 --- a/ui/js/src/kimchi.guest_main.js +++ b/ui/js/src/kimchi.guest_main.js @@ -201,6 +201,21 @@ kimchi.listVmsAuto = function() { if (kimchi.vmTimeout) { clearTimeout(kimchi.vmTimeout); } + var getCreatingGuests = function(){ + var guests = []; + kimchi.getTasksByFilter('status=running&target_uri='+encodeURIComponent('^/vms/[^/]+$'), function(tasks) { + for(var i=0;i<tasks.length;i++){ + var guestUri = tasks[i].target_uri; + var guestName = guestUri.split('/')[1] + guests.push($.extend({}, kimchi.sampleGuestObject, {name: guestName, isCreating: true})); + if(kimchi.trackingTasks.indexOf(tasks[i].id)==-1) + kimchi.trackTask(tasks[i].id, null, function(err){ + kimchi.message.error(err.message); + }, null); + } + }, null, true); + return guests; + }; var getCloningGuests = function(){ var guests = []; kimchi.getTasksByFilter('status=running&target_uri='+encodeURIComponent('^/vms/.+/clone'), function(tasks) { @@ -219,6 +234,7 @@ kimchi.listVmsAuto = function() { kimchi.listVMs(function(result, textStatus, jqXHR) { if (result && textStatus=="success") { result = getCloningGuests().concat(result); + result = getCreatingGuests().concat(result); if(result.length) { var listHtml = ''; var guestTemplate = kimchi.guestTemplate; @@ -281,7 +297,7 @@ kimchi.createGuestLi = function(vmObject, prevScreenImage, openMenu) { imgLoad.attr('src',load_src); //Link the stopped tile to the start action, the running tile to open the console - if(!vmObject.isCloning){ + if(!(vmObject.isCloning || vmObject.isCreating)){ if (vmRunningBool) { liveTile.off("click", kimchi.vmstart); liveTile.on("click", kimchi.openVmConsole); @@ -329,7 +345,7 @@ kimchi.createGuestLi = function(vmObject, prevScreenImage, openMenu) { } //Setup action event handlers - if(!vmObject.isCloning){ + if(!(vmObject.isCloning || vmObject.isCreating)){ guestActions.find("[name=vm-start]").on({click : kimchi.vmstart}); guestActions.find("[name=vm-poweroff]").on({click : kimchi.vmpoweroff}); if (vmRunningBool) { //If the guest is not running, do not enable reset @@ -362,7 +378,10 @@ kimchi.createGuestLi = function(vmObject, prevScreenImage, openMenu) { }else{ guestActions.find('.btn').attr('disabled', true); - result.find('.guest-clone').removeClass('hide-content'); + if(vmObject.isCloning) + result.find('.guest-clone').removeClass('hide-content'); + else + result.find('.guest-create').removeClass('hide-content'); $('.popover', guestActions.find("div[name=actionmenu]")).remove(); } diff --git a/ui/pages/guest.html.tmpl b/ui/pages/guest.html.tmpl index 17d41ac..aaf41a2 100644 --- a/ui/pages/guest.html.tmpl +++ b/ui/pages/guest.html.tmpl @@ -29,6 +29,9 @@ <div class="guest-clone hide-content"> <span class="icon"></span><span class="text">$_("Cloning")...</span> </div> + <div class="guest-create hide-content"> + <span class="icon"></span><span class="text">$_("Creating")...</span> + </div> </div> <div name="cpu_utilization" class="sortable"> <div class="circleGauge"></div> -- 2.1.0

1 0

[PATCH 4/6] Create VMs asynchronously: Backend
by Christy Perez 28 Apr '15

28 Apr '15

Signed-off-by: Christy Perez <christy(a)linux.vnet.ibm.com> --- src/kimchi/control/vms.py | 4 ++-- src/kimchi/model/vms.py | 27 ++++++++++++++++++++++++--- 2 files changed, 26 insertions(+), 5 deletions(-) diff --git a/src/kimchi/control/vms.py b/src/kimchi/control/vms.py index 6352a26..a40b56e 100644 --- a/src/kimchi/control/vms.py +++ b/src/kimchi/control/vms.py @@ -17,13 +17,13 @@ # License along with this library; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA -from kimchi.control.base import Collection, Resource +from kimchi.control.base import AsyncCollection, Resource from kimchi.control.utils import internal_redirect, UrlSubNode from kimchi.control.vm import sub_nodes @UrlSubNode('vms', True) -class VMs(Collection): +class VMs(AsyncCollection): def __init__(self, model): super(VMs, self).__init__(model) self.resource = VM diff --git a/src/kimchi/model/vms.py b/src/kimchi/model/vms.py index 5f8c3da..9786063 100644 --- a/src/kimchi/model/vms.py +++ b/src/kimchi/model/vms.py @@ -79,9 +79,9 @@ def __init__(self, **kargs): self.conn = kargs['conn'] self.objstore = kargs['objstore'] self.caps = CapabilitiesModel(**kargs) + self.task = TaskModel(**kargs) def create(self, params): - conn = self.conn.get() t_name = template_name_from_uri(params['template']) vm_uuid = str(uuid.uuid4()) vm_list = self.get_list() @@ -102,7 +102,26 @@ def create(self, params): raise InvalidOperation("KCHVM0005E") t.validate() + taskid = add_task(u'/vms/%s' % name, + self._create_task, self.objstore, + {'vm_uuid': vm_uuid, 'template': t, 'name': name}) + + return self.task.lookup(taskid) + + def _create_task(self, cb, params): + """ + params: A dict with the following values: + - vm_uuid: The UUID of the VM being created + - template: The template being used to create the VM + - name: The name for the new VM + """ + + vm_uuid = params['vm_uuid'] + t = params['template'] + name = params['name'] + conn = self.conn.get() + cb('Storing VM icon') # Store the icon for displaying later icon = t.info.get('icon') if icon: @@ -117,6 +136,7 @@ def create(self, params): # If storagepool is SCSI, volumes will be LUNs and must be passed by # the user from UI or manually. + cb('Provisioning storage for new VM') vol_list = [] if t._get_storage_type() not in ["iscsi", "scsi"]: vol_list = t.fork_vm_storage(vm_uuid) @@ -128,6 +148,7 @@ def create(self, params): graphics=graphics, volumes=vol_list) + cb('Defining new VM') try: conn.defineXML(xml.encode('utf-8')) except libvirt.libvirtError as e: @@ -138,10 +159,10 @@ def create(self, params): raise OperationFailed("KCHVM0007E", {'name': name, 'err': e.get_error_message()}) + cb('Updating VM metadata') VMModel.vm_update_os_metadata(VMModel.get_vm(name, self.conn), t.info, self.caps.metadata_support) - - return name + cb('OK', True) def get_list(self): return VMsModel.get_vms(self.conn) -- 2.1.0

1 0

[PATCH 3/6] UI changes for new clone target_uri
by Christy Perez 28 Apr '15

28 Apr '15

Signed-off-by: Christy Perez <christy(a)linux.vnet.ibm.com> --- ui/js/src/kimchi.guest_main.js | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ui/js/src/kimchi.guest_main.js b/ui/js/src/kimchi.guest_main.js index b66177c..c712bb4 100644 --- a/ui/js/src/kimchi.guest_main.js +++ b/ui/js/src/kimchi.guest_main.js @@ -203,10 +203,10 @@ kimchi.listVmsAuto = function() { } var getCloningGuests = function(){ var guests = []; - kimchi.getTasksByFilter('status=running&target_uri='+encodeURIComponent('^/vms/*'), function(tasks) { + kimchi.getTasksByFilter('status=running&target_uri='+encodeURIComponent('^/vms/.+/clone'), function(tasks) { for(var i=0;i<tasks.length;i++){ var guestUri = tasks[i].target_uri; - var guestName = guestUri.substring(guestUri.lastIndexOf('/')+1, guestUri.length); + var guestName = guestUri.split('/')[2] guests.push($.extend({}, kimchi.sampleGuestObject, {name: guestName, isCloning: true})); if(kimchi.trackingTasks.indexOf(tasks[i].id)==-1) kimchi.trackTask(tasks[i].id, null, function(err){ -- 2.1.0

1 0

[PATCH v5 0/6] Async VM Creation
by Christy Perez 28 Apr '15

28 Apr '15

v5 Changes: - Rebase - Remove all the unimperative target_uri changes (see below) If a guest has a large disk, and uses a filesystem that requires preallocation, it can take several minutes to create a VM. During that time, kimchi is tied up by the VM creation. This patch changes the VMs Collection to be an AsyncCollection. Another change required for this was to create a more granular way to query vm-related tasks. The original idea was to add another field to the task database, but then Aline suggested just modifying the task_uri. Since the task_uri cretation will be changed in a future patchset, this is now only modified for the conflicting (clone) tasks. Christy Perez (6): Append clone to target_uri for vm clone task Tests for new clone target_uri UI changes for new clone target_uri Create VMs asynchronously: Backend Create VMs Asynchronously: Tests Create VMs Asynchronously: UI src/kimchi/control/vms.py | 4 +-- src/kimchi/model/vms.py | 32 +++++++++++++++++---- tests/test_authorization.py | 23 +++++++++------ tests/test_mockmodel.py | 12 ++++++-- tests/test_model.py | 49 +++++++++++++++++++++----------- tests/test_model_storagevolume.py | 2 +- tests/test_rest.py | 60 +++++++++++++++++++++++++++++---------- ui/css/theme-default/list.css | 18 ++++++++++++ ui/js/src/kimchi.guest_main.js | 29 +++++++++++++++---- ui/pages/guest.html.tmpl | 3 ++ 10 files changed, 175 insertions(+), 57 deletions(-) -- 2.1.0

2 9

[PATCH] Improve code to retrieve the number of host CPUs
by Jose Ricardo Ziviani 28 Apr '15

28 Apr '15

- Retrieving the number of cpus using psinfo is unstable, there are three (at least) different ways to do it that changes from version to version. This code improves the way such functions are called to avoid any surprise in future. Signed-off-by: Jose Ricardo Ziviani <joserz(a)linux.vnet.ibm.com> --- src/kimchi/model/host.py | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/src/kimchi/model/host.py b/src/kimchi/model/host.py index 293c4ce..b2fa379 100644 --- a/src/kimchi/model/host.py +++ b/src/kimchi/model/host.py @@ -102,15 +102,20 @@ class HostModel(object): return res def lookup(self, *name): - cpus = 0 + cpus = psutil.NUM_CPUS - # Only newer psutil versions have a portable method to get - # the number of cpus - try: + # psutil is unstable on how to get the number of + # cpus, different versions call it differently + if hasattr(psutil, 'cpu_count'): cpus = psutil.cpu_count() - except AttributeError: - cpus = psutil._psplatform.get_num_cpus() + elif hasattr(psutil, '_psplatform'): + for method_name in ['_get_num_cpus', 'get_num_cpus']: + + method = getattr(psutil._psplatform, method_name, None) + if method is not None: + cpus = method() + break self.host_info['cpus'] = cpus self.host_info['memory'] = psutil.phymem_usage().total -- 1.9.1

2 2

[PATCH] Move kimchi nginx config file to nginx default directory.
by Jose Ricardo Ziviani 28 Apr '15

28 Apr '15

- Move Kimchi nginx config file to /etc/nginx/conf.d. - Rename nginx_kimchi.conf to kimchi.conf. --- .gitignore | 2 +- configure.ac | 1 + contrib/kimchi.spec.fedora.in | 6 ++-- contrib/kimchi.spec.suse.in | 7 ++-- src/Makefile.am | 5 ++- src/kimchi/config.py.in | 2 ++ src/kimchi/proxy.py | 12 +++---- src/nginx.conf.in | 76 ------------------------------------------- src/nginx/Makefile.am | 23 +++++++++++++ src/nginx/kimchi.conf.in | 76 +++++++++++++++++++++++++++++++++++++++++++ 10 files changed, 119 insertions(+), 91 deletions(-) delete mode 100644 src/nginx.conf.in create mode 100644 src/nginx/Makefile.am create mode 100644 src/nginx/kimchi.conf.in diff --git a/.gitignore b/.gitignore index 4abfc0a..a318bd9 100644 --- a/.gitignore +++ b/.gitignore @@ -28,7 +28,7 @@ stamp-po kimchi-*.tar.gz src/kimchid src/kimchi.conf -src/nginx_kimchi.conf +src/nginx/kimchi.conf src/kimchi/config.py tests/run_tests.sh tests/test_config.py diff --git a/configure.ac b/configure.ac index 1b476c3..5da3240 100644 --- a/configure.ac +++ b/configure.ac @@ -80,6 +80,7 @@ AC_CONFIG_FILES([ docs/Makefile src/Makefile src/distros.d/Makefile + src/nginx/Makefile src/kimchi/Makefile src/kimchi/control/Makefile src/kimchi/control/vm/Makefile diff --git a/contrib/kimchi.spec.fedora.in b/contrib/kimchi.spec.fedora.in index 750dada..a721a7f 100644 --- a/contrib/kimchi.spec.fedora.in +++ b/contrib/kimchi.spec.fedora.in @@ -94,7 +94,7 @@ touch %{buildroot}/%{_localstatedir}/log/kimchi/kimchi-error.log # create /etc/kimchi structure mkdir -p %{buildroot}/%{_sysconfdir}/kimchi/ -touch %{buildroot}/%{_sysconfdir}/kimchi/nginx_kimchi.conf +touch %{buildroot}/%{_sysconfdir}/nginx/conf.d/kimchi.conf # Install the systemd scripts install -Dm 0644 contrib/kimchid.service.fedora %{buildroot}%{_unitdir}/kimchid.service @@ -158,14 +158,14 @@ rm -rf $RPM_BUILD_ROOT %{_datadir}/kimchi/config/ui/*.xml %{_datadir}/kimchi/ui/ %{_datadir}/kimchi +%{_sysconfdir}/nginx/conf.d/kimchi.conf.in +%{_sysconfdir}/nginx/conf.d/kimchi.conf %{_sysconfdir}/kimchi/kimchi.conf -%{_sysconfdir}/kimchi/nginx.conf.in %{_sysconfdir}/kimchi/distros.d/debian.json %{_sysconfdir}/kimchi/distros.d/fedora.json %{_sysconfdir}/kimchi/distros.d/opensuse.json %{_sysconfdir}/kimchi/distros.d/ubuntu.json %{_sysconfdir}/kimchi/distros.d/gentoo.json -%{_sysconfdir}/kimchi/nginx_kimchi.conf %{_sysconfdir}/kimchi/ %{_sharedstatedir}/kimchi/debugreports/ %{_sharedstatedir}/kimchi/screenshots/ diff --git a/contrib/kimchi.spec.suse.in b/contrib/kimchi.spec.suse.in index 7e4172d..5673ced 100644 --- a/contrib/kimchi.spec.suse.in +++ b/contrib/kimchi.spec.suse.in @@ -63,7 +63,9 @@ touch %{buildroot}/%{_localstatedir}/log/kimchi/kimchi-error.log # create /etc/kimchi structure mkdir -p %{buildroot}/%{_sysconfdir}/kimchi/ -touch %{buildroot}/%{_sysconfdir}/kimchi/nginx_kimchi.conf + +# create nginx configuration +touch %{buildroot}/%{_sysconfdir}/nginx/conf.d/kimchi.conf # Install the SysV init scripts install -Dm 0755 contrib/kimchid.sysvinit %{buildroot}%{_initrddir}/kimchid @@ -97,13 +99,14 @@ rm -rf $RPM_BUILD_ROOT %{_datadir}/kimchi/config/ui/*.xml %{_datadir}/kimchi/ui/ %{_datadir}/kimchi +%{_sysconfdir}/nginx/conf.d/kimchi.conf.in +%{_sysconfdir}/nginx/conf.d/kimchi.conf %{_sysconfdir}/kimchi/kimchi.conf %{_sysconfdir}/kimchi/distros.d/debian.json %{_sysconfdir}/kimchi/distros.d/fedora.json %{_sysconfdir}/kimchi/distros.d/opensuse.json %{_sysconfdir}/kimchi/distros.d/ubuntu.json %{_sysconfdir}/kimchi/distros.d/gentoo.json -%{_sysconfdir}/kimchi/nginx_kimchi.conf %{_sysconfdir}/kimchi %{_initrddir}/kimchid %{_sysconfdir}/kimchi/ diff --git a/src/Makefile.am b/src/Makefile.am index dfeb24e..edc163e 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -17,18 +17,17 @@ # License along with this library; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA -SUBDIRS = kimchi distros.d +SUBDIRS = kimchi distros.d nginx EXTRA_DIST = kimchid.in \ kimchi.conf.in \ - nginx.conf.in \ firewalld.xml \ $(NULL) bin_SCRIPTS = kimchid confdir = $(sysconfdir)/kimchi -dist_conf_DATA = kimchi.conf nginx.conf.in +dist_conf_DATA = kimchi.conf BUILT_SOURCES = kimchi.conf diff --git a/src/kimchi/config.py.in b/src/kimchi/config.py.in index f2e1cac..f15d228 100644 --- a/src/kimchi/config.py.in +++ b/src/kimchi/config.py.in @@ -112,6 +112,7 @@ class Paths(object): self.novnc_dir = '/usr/share/novnc' if self.installed: + self.nginx_conf_dir = '@sysconfdir@/nginx/conf.d' self.state_dir = '@localstatedir@/lib/kimchi' self.log_dir = '@localstatedir@/log/kimchi' self.conf_dir = '@sysconfdir@/kimchi' @@ -120,6 +121,7 @@ class Paths(object): self.mo_dir = '@prefix@/share/locale' self.spice_css_file = os.path.join(self.spice_dir, 'spice.css') else: + self.nginx_conf_dir = self.add_prefix('src/nginx') self.state_dir = self.add_prefix('data') self.log_dir = self.add_prefix('log') self.conf_dir = self.add_prefix('src') diff --git a/src/kimchi/proxy.py b/src/kimchi/proxy.py index fafa5bc..1b70d5e 100644 --- a/src/kimchi/proxy.py +++ b/src/kimchi/proxy.py @@ -37,8 +37,7 @@ def _create_proxy_config(options): To allow flexibility in which port kimchi runs, we need the same flexibility with the nginx proxy. This method creates the config file dynamically by using 'nginx.conf.in' as a template, creating - the file 'nginx_kimchi.config' which will be used to launch the - proxy. + the file 'kimchi.conf' which will be used to launch the proxy. Arguments: options - OptionParser object with Kimchi config options @@ -53,6 +52,7 @@ def _create_proxy_config(options): user_proxy = 'www-data' config_dir = paths.conf_dir + nginx_config_dir = paths.nginx_conf_dir cert = options.ssl_cert key = options.ssl_key @@ -70,7 +70,7 @@ def _create_proxy_config(options): # Read template file and create a new config file # with the specified parameters. - with open(os.path.join(config_dir, "nginx.conf.in")) as template: + with open(os.path.join(nginx_config_dir, "kimchi.conf.in")) as template: data = template.read() data = Template(data) data = data.safe_substitute(user=user_proxy, @@ -81,7 +81,7 @@ def _create_proxy_config(options): max_body_size=eval(options.max_body_size)) # Write file to be used for nginx. - config_file = open(os.path.join(config_dir, "nginx_kimchi.conf"), "w") + config_file = open(os.path.join(nginx_config_dir, "kimchi.conf"), "w") config_file.write(data) config_file.close() @@ -89,8 +89,8 @@ def _create_proxy_config(options): def start_proxy(options): """Start nginx reverse proxy.""" _create_proxy_config(options) - config_dir = paths.conf_dir - config_file = "%s/nginx_kimchi.conf" % config_dir + nginx_config_dir = paths.nginx_conf_dir + config_file = "%s/kimchi.conf" % nginx_config_dir cmd = ['nginx', '-c', config_file] subprocess.call(cmd) diff --git a/src/nginx.conf.in b/src/nginx.conf.in deleted file mode 100644 index e308152..0000000 --- a/src/nginx.conf.in +++ /dev/null @@ -1,76 +0,0 @@ -# Project Kimchi -# -# Copyright IBM, Corp. 2014 -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library; if not, write to the Free Software -# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA -# 02110-1301 USA - -# This is a template file to be used to generate a nginx -# proxy config file at kimchid script. - -user ${user}; -worker_processes 1; - -error_log /var/log/nginx/error.log; - -events { - worker_connections 1024; -} - -http { - - log_format main '$remote_addr - $remote_user [$time_local] "$request" ' - '$status $body_bytes_sent "$http_referer" ' - '"$http_user_agent" "$http_x_forwarded_for"'; - - access_log /var/log/nginx/access.log main; - sendfile on; - - client_max_body_size ${max_body_size}k; - - # Timeout set to 10 minutes to avoid the 504 Gateway Timeout - # when Kimchi is processing a request. - proxy_connect_timeout 600; - proxy_send_timeout 600; - proxy_read_timeout 600; - send_timeout 600; - - server { - listen ${proxy_ssl_port} ssl; - - ssl_certificate ${cert_pem}; - ssl_certificate_key ${cert_key}; - ssl_protocols TLSv1 TLSv1.1 TLSv1.2; - ssl_ciphers ECDH@STRENGTH:DH@STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNULL; - - add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;"; - add_header X-Frame-Options DENY; - add_header X-Content-Type-Options nosniff; - add_header X-XSS-Protection "1; mode=block"; - - location / { - proxy_pass http://127.0.0.1:${kimchid_port}; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_redirect http://127.0.0.1:${kimchid_port}/ https://$host:${proxy_ssl_port}/; - } - } - - server { - listen ${proxy_port}; - rewrite ^/(.*)$ https://$host:${proxy_ssl_port}/$1 redirect; - } -} diff --git a/src/nginx/Makefile.am b/src/nginx/Makefile.am new file mode 100644 index 0000000..b240508 --- /dev/null +++ b/src/nginx/Makefile.am @@ -0,0 +1,23 @@ +# +# Kimchi +# +# Copyright IBM Corp, 2013 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +EXTRA_DIST = kimchi.conf.in + +confdir = $(sysconfdir)/nginx/conf.d +dist_conf_DATA = kimchi.conf.in diff --git a/src/nginx/kimchi.conf.in b/src/nginx/kimchi.conf.in new file mode 100644 index 0000000..e308152 --- /dev/null +++ b/src/nginx/kimchi.conf.in @@ -0,0 +1,76 @@ +# Project Kimchi +# +# Copyright IBM, Corp. 2014 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301 USA + +# This is a template file to be used to generate a nginx +# proxy config file at kimchid script. + +user ${user}; +worker_processes 1; + +error_log /var/log/nginx/error.log; + +events { + worker_connections 1024; +} + +http { + + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; + + access_log /var/log/nginx/access.log main; + sendfile on; + + client_max_body_size ${max_body_size}k; + + # Timeout set to 10 minutes to avoid the 504 Gateway Timeout + # when Kimchi is processing a request. + proxy_connect_timeout 600; + proxy_send_timeout 600; + proxy_read_timeout 600; + send_timeout 600; + + server { + listen ${proxy_ssl_port} ssl; + + ssl_certificate ${cert_pem}; + ssl_certificate_key ${cert_key}; + ssl_protocols TLSv1 TLSv1.1 TLSv1.2; + ssl_ciphers ECDH@STRENGTH:DH@STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNULL; + + add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;"; + add_header X-Frame-Options DENY; + add_header X-Content-Type-Options nosniff; + add_header X-XSS-Protection "1; mode=block"; + + location / { + proxy_pass http://127.0.0.1:${kimchid_port}; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_redirect http://127.0.0.1:${kimchid_port}/ https://$host:${proxy_ssl_port}/; + } + } + + server { + listen ${proxy_port}; + rewrite ^/(.*)$ https://$host:${proxy_ssl_port}/$1 redirect; + } +} -- 1.9.1

2 2

Re: [Kimchi-devel] [PATCH 3/5] Use get_vm() instead of conn.lookupByName
by Ramon Medeiros 28 Apr '15

28 Apr '15

Please ignore these patches On 04/28/2015 10:16 AM, Ramon Medeiros wrote: > Remove duplicated code of conn.lookupByName. > > Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> > --- > src/kimchi/model/vms.py | 2 +- > src/kimchi/model/vmstorages.py | 4 ++-- > 2 files changed, 3 insertions(+), 3 deletions(-) > > diff --git a/src/kimchi/model/vms.py b/src/kimchi/model/vms.py > index 6792447..87d63c1 100644 > --- a/src/kimchi/model/vms.py > +++ b/src/kimchi/model/vms.py > @@ -266,7 +266,7 @@ class VMModel(object): > # fetch base XML > cb('reading source VM XML') > try: > - vir_dom = vir_conn.lookupByName(name) > + vir_dom = self.get_vm(name) > flags = libvirt.VIR_DOMAIN_XML_SECURE > xml = vir_dom.XMLDesc(flags).decode('utf-8') > except libvirt.libvirtError, e: > diff --git a/src/kimchi/model/vmstorages.py b/src/kimchi/model/vmstorages.py > index 37aca64..8dabae7 100644 > --- a/src/kimchi/model/vmstorages.py > +++ b/src/kimchi/model/vmstorages.py > @@ -138,7 +138,7 @@ class VMStoragesModel(object): > dev, xml = get_disk_xml(params) > try: > conn = self.conn.get() > - dom = conn.lookupByName(vm_name) > + dom = VMModel.get_vm(vm_name, conn) > dom.attachDeviceFlags(xml, get_vm_config_flag(dom, 'all')) > except Exception as e: > raise OperationFailed("KCHVMSTOR0008E", {'error': e.message}) > @@ -173,7 +173,7 @@ class VMStorageModel(object): > > try: > bus_type = self.lookup(vm_name, dev_name)['bus'] > - dom = conn.lookupByName(vm_name) > + dom = VMModel.get_vm(vm_name, conn) > except NotFoundError: > raise > -- Ramon Nunes Medeiros Kimchi Developer Linux Technology Center Brazil IBM Systems & Technology Group Phone : +55 19 2132 7878 ramonn(a)br.ibm.com

1 0

[PATCH] Fix disk format lock and checking during template creation
by Rodrigo Trujillo 27 Apr '15

27 Apr '15

Users are able to pass the disk format (qcow, raw, etc) in disk Template information. However, Kimchi was ignoring this information and always creating QCOW2 disk images (if the storagepool is not 'LOGICAL') when it creates a VM based on a given Template. This patch fixes this problem and also moves the checking and disk format auto assignment to Template creation phase, in VMTemplate class init. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/kimchi/i18n.py | 1 + src/kimchi/vmtemplate.py | 15 +++++++++++---- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/src/kimchi/i18n.py b/src/kimchi/i18n.py index 18e84bc..0651032 100644 --- a/src/kimchi/i18n.py +++ b/src/kimchi/i18n.py @@ -154,6 +154,7 @@ messages = { "KCHTMPL0025E": _("When specifying CPU topology, VCPUs must be a product of sockets, cores, and threads."), "KCHTMPL0026E": _("When specifying CPU topology, each element must be an integer greater than zero."), "KCHTMPL0027E": _("Invalid disk image format. Valid formats: bochs, cloop, cow, dmg, qcow, qcow2, qed, raw, vmdk, vpc."), + "KCHTMPL0028E": _("Invalid disk image format. Logical, SCSI and iSCSI storage pools must use disk image format 'raw'."), "KCHPOOL0001E": _("Storage pool %(name)s already exists"), "KCHPOOL0002E": _("Storage pool %(name)s does not exist"), diff --git a/src/kimchi/vmtemplate.py b/src/kimchi/vmtemplate.py index ef97d83..84e877b 100644 --- a/src/kimchi/vmtemplate.py +++ b/src/kimchi/vmtemplate.py @@ -75,8 +75,16 @@ class VMTemplate(object): args['graphics'] = graphics self.info.update(args) - # Assign right disk format to logical and [i]scsi storagepools + # Check disk image format passed and pool type if self._get_storage_type() in ['logical', 'iscsi', 'scsi']: + paramdisks = args.get('disks') + if paramdisks is not None: + for disk in paramdisks: + fmt = disk.get('format') + if fmt is not None and fmt != 'raw': + raise InvalidParameter('KCHTMPL0028E') + # Finally add disk image format when not passed or + # using default for i, disk in enumerate(self.info['disks']): self.info['disks'][i]['format'] = 'raw' @@ -198,7 +206,6 @@ class VMTemplate(object): def to_volume_list(self, vm_uuid): storage_path = self._get_storage_path() - fmt = 'raw' if self._get_storage_type() in ['logical'] else 'qcow2' ret = [] for i, d in enumerate(self.info['disks']): index = d.get('index', i) @@ -206,11 +213,11 @@ class VMTemplate(object): info = {'name': volume, 'capacity': d['size'], - 'format': fmt, + 'format': d['format'], 'path': '%s/%s' % (storage_path, volume)} if 'logical' == self._get_storage_type() or \ - fmt not in ['qcow2', 'raw']: + info['format'] not in ['qcow2', 'raw']: info['allocation'] = info['capacity'] else: info['allocation'] = 0 -- 2.1.0

2 3

[PATCH V2] UI-Template Edit: Enable user to change disk format
by Rodrigo Trujillo 27 Apr '15

27 Apr '15

This patch adds a new field (Disk Format) in Storage tab in Template edit window. Users will then be allowed to select any disk format supported by Libvirt and Kimchi backend. The default disk format is qcow2 (first option), so, for compatibility, if the template register in objecstore does not contain the disk format information, the new field is going to set 'qcow2' automatically. For iscsi storagepools, the new field is going to be disabled and the format is going to be set automatically as 'raw', just like the backend behaves currently, avoiding errors. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- ui/css/theme-default/template-edit.css | 8 ++++-- ui/js/src/kimchi.template_edit_main.js | 52 ++++++++++++++++++++++++++++++---- ui/pages/template-edit.html.tmpl | 16 +++++++++++ 3 files changed, 68 insertions(+), 8 deletions(-) diff --git a/ui/css/theme-default/template-edit.css b/ui/css/theme-default/template-edit.css index 7abee7c..ed9f02c 100644 --- a/ui/css/theme-default/template-edit.css +++ b/ui/css/theme-default/template-edit.css @@ -95,7 +95,7 @@ #edit-template-tabs .template-storage-cell{ display: inline-block; - width: 230px; + width: 180px; } #edit-template-tabs .template-storage-cell label { @@ -114,7 +114,11 @@ } #form-template-storage .template-tab-body .template-storage-name { - width: 220px; + width: 170px; +} + +#form-template-storage .template-tab-body .template-storage-disk-format { + width: 160px; } #edit-template-tabs .template-tab-body input[readonly] { diff --git a/ui/js/src/kimchi.template_edit_main.js b/ui/js/src/kimchi.template_edit_main.js index 85f76cd..6029a95 100644 --- a/ui/js/src/kimchi.template_edit_main.js +++ b/ui/js/src/kimchi.template_edit_main.js @@ -67,7 +67,7 @@ kimchi.template_edit_main = function() { $('.template-tab-body', '#form-template-storage').append(nodeStorage); var storageOptions = ''; var scsiOptions = ''; - $('select', '#form-template-storage').find('option').remove(); + $('select:first', '#form-template-storage').find('option').remove(); $.each(result, function(index, storageEntities) { if((storageEntities.state === 'active') && (storageEntities.type != 'kimchi-iso')) { if(storageEntities.type === 'iscsi' || storageEntities.type === 'scsi') { @@ -77,7 +77,7 @@ kimchi.template_edit_main = function() { var isSlected = tmpPath === thisName ? ' selected' : ''; scsiOptions += '<option' + isSlected + '>' + tmpPath + '</option>'; }); - $('select', '#form-template-storage').append(scsiOptions); + $('select:first', '#form-template-storage').append(scsiOptions); }, function() {}); } else { var isSlected = storageEntities.name === thisName ? ' selected' : ''; @@ -85,8 +85,20 @@ kimchi.template_edit_main = function() { } } }); - $('select', '#form-template-storage').append(storageOptions); - $('select', '#form-template-storage').change(function() { + $('select:first', '#form-template-storage').append(storageOptions); + + // Set disk format + $('select:last option', '#form-template-storage').each(function() { + if ($(this).text() == storageData.storageDiskFormat) { + $(this).prop('selected', true); + } + }); + + $('select:last', '#form-template-storage').change(function() { + $('.template-storage-disk-format').val($(this).val()); + }); + + $('select:first', '#form-template-storage').change(function() { var selectedItem = $(this).parent().parent(); var tempStorageNameFull = $(this).val(); var tempName = tempStorageNameFull.split('/'); @@ -99,9 +111,25 @@ kimchi.template_edit_main = function() { kimchi.getStoragePoolVolume(tempStorageName, tempName[tempName.length-1], function(info) { volSize = info.capacity / Math.pow(1024, 3); $('.template-storage-disk', selectedItem).attr('readonly', true).val(volSize); + $('select:last option', selectedItem).each(function() { + this.selected = (this.text == 'raw'); + }); + $('select:last', selectedItem).prop('disabled', true).change(); }); + } else if (tempType === 'logical') { + $('.template-storage-disk', selectedItem).attr('readonly', false); + $('select:last option', selectedItem).each(function() { + this.selected = (this.text == 'raw'); + }); + $('select:last', selectedItem).prop('disabled', true).change(); } else { $('.template-storage-disk', selectedItem).attr('readonly', false); + if ($('select:last', selectedItem).prop('disabled') == true) { + $('select:last option', selectedItem).each(function() { + this.selected = (this.text == 'qcow2'); + }); + $('select:last', selectedItem).prop('disabled', false).change(); + } } }); }); @@ -120,7 +148,8 @@ kimchi.template_edit_main = function() { editMode : 'hide', storageName : defaultPool, storageType : defaultType, - storageDisk : diskEntities.size + storageDisk : diskEntities.size, + storageDiskFormat : diskEntities.format ? diskEntities.format : 'qcow2' } if (diskEntities.volume) { @@ -131,7 +160,17 @@ kimchi.template_edit_main = function() { nodeData.storageDisk = volSize; addStorageItem(nodeData); $('.template-storage-disk').attr('readonly', true); + $('select:last option', '#form-template-storage').each(function() { + this.selected = (this.text == 'raw'); + }); + $('select:last', '#form-template-storage').prop('disabled', true).change(); + }); + } else if (defaultType === 'logical') { + addStorageItem(storageNodeData); + $('select:last option', '#form-template-storage').each(function() { + this.selected = (this.text == 'raw'); }); + $('select:last', '#form-template-storage').prop('disabled', true).change(); } else { addStorageItem(storageNodeData); } @@ -271,7 +310,8 @@ kimchi.template_edit_main = function() { origDisks[0]['volume'] && delete origDisks[0]['volume']; origDisks[0].size = Number($('.template-storage-disk', tmpItem).val()); } - data[field] = origDisks; + origDisks[0].format = $('.template-storage-disk-format', tmpItem).val(); + data[field] = origDisks; } else if (field == 'graphics') { var type = $('#form-template-general [name="' + field + '"]').val(); diff --git a/ui/pages/template-edit.html.tmpl b/ui/pages/template-edit.html.tmpl index c7832c9..e64a30f 100644 --- a/ui/pages/template-edit.html.tmpl +++ b/ui/pages/template-edit.html.tmpl @@ -104,6 +104,7 @@ <span class="template-storage-cell">$_("Storage Pool")</span> <span class="template-storage-cell">$_("Type")</span> <span class="template-storage-cell">$_("Disk(GB)")</span> + <span class="template-storage-cell">$_("Disk Format")</span> <button type="button" id="template-edit-storage-add-button" class="action-area"></button> </div> <div class="template-tab-body"> @@ -160,6 +161,21 @@ <span class="template-storage-cell"> <input class="template-storage-disk" value={storageDisk} type="text" /> </span> + <span class="template-storage-cell"> + <input class="template-storage-disk-format" value={storageDiskFormat} type="text" style="display:none" /> + <select> + <option>qcow2</option> + <option>raw</option> + <option>bochs</option> + <option>cloop</option> + <option>cow</option> + <option>dmg</option> + <option>qcow</option> + <option>qed</option> + <option>vmdk</option> + <option>vpc</option> + </select> + </span> </div> </script> <script id="template-interface-tmpl" type="text/html"> -- 2.1.0

2 3

[PATCH V3 0/2] Warn users about NetworkManager instability
by Jose Ricardo Ziviani 27 Apr '15

27 Apr '15

V3: - Use cached capability value. V2: - Change is_nm_running to use a 'cross distro' command. - Update the warning message NetworkManager might have some instability when managing the bridges created by libvirt. There some bugs against NM about it and there is no much we can do more than warn the users about it. So, this patchset checks if NM is running and, if so, will display a message to users. Jose Ricardo Ziviani (2): Implement function to check if NM is running. Add a warn about NM running in the system. src/kimchi/model/config.py | 1 + src/kimchi/model/featuretests.py | 10 ++++++++++ tests/test_rest.py | 3 ++- ui/js/src/kimchi.network.js | 3 +++ ui/pages/i18n.json.tmpl | 1 + 5 files changed, 17 insertions(+), 1 deletion(-) -- 1.9.1

2 3

[PATCH] Allow creating VM storages with ref_cnt > 0
by Crístian Deives 27 Apr '15

27 Apr '15

As discussed in an e-mail thread (http://lists.ovirt.org/pipermail/kimchi-devel/2015-March/010154.html) we should stop blocking the user from creating VM storages when the storage has already been added to some VM. The UI may display a warning when that cases happens so the user should decide what to do. Remove check on ref_cnt when creating a VM storage. Signed-off-by: Crístian Deives <cristiandeives(a)gmail.com> --- src/kimchi/i18n.py | 1 - src/kimchi/model/vmstorages.py | 2 -- 2 files changed, 3 deletions(-) diff --git a/src/kimchi/i18n.py b/src/kimchi/i18n.py index 18e84bc..6b4726c 100644 --- a/src/kimchi/i18n.py +++ b/src/kimchi/i18n.py @@ -285,7 +285,6 @@ messages = { "KCHVMSTOR0013E": _("Specify path to update virtual machine disk"), "KCHVMSTOR0014E": _("Controller type %(type)s limitation of %(limit)s devices reached"), "KCHVMSTOR0015E": _("Cannot retrieve disk path information for given pool/volume: %(error)s"), - "KCHVMSTOR0016E": _("Volume already in use by other virtual machine."), "KCHVMSTOR0017E": _("Only one of path or pool/volume can be specified to add a new virtual machine disk"), "KCHVMSTOR0018E": _("Volume chosen with format %(format)s does not fit in the storage type %(type)s"), diff --git a/src/kimchi/model/vmstorages.py b/src/kimchi/model/vmstorages.py index 37aca64..0375772 100644 --- a/src/kimchi/model/vmstorages.py +++ b/src/kimchi/model/vmstorages.py @@ -111,8 +111,6 @@ class VMStoragesModel(object): raise InvalidParameter("KCHVMSTOR0012E") except Exception as e: raise InvalidParameter("KCHVMSTOR0015E", {'error': e}) - if vol_info['ref_cnt'] != 0: - raise InvalidParameter("KCHVMSTOR0016E") valid_format = { "disk": ["raw", "bochs", "qcow", "qcow2", "qed", "vmdk"], -- 2.1.0

2 1

[PATCH 0/3] Warn users about NetworkManager instability
by Jose Ricardo Ziviani 27 Apr '15

27 Apr '15

NetworkManager might have some instability when managing the bridges created by libvirt. There some bugs against NM about it and there is no much we can do more than warn the users about it. So, this patchset checks if NM is running and, if so, will display a message to users. Jose Ricardo Ziviani (3): Add the env parameter to run_command. Implement function to check if NM is running. Add a warn about NM running in the system. src/kimchi/model/config.py | 1 + src/kimchi/model/featuretests.py | 28 ++++++++++++++++++++++++++++ src/kimchi/utils.py | 6 ++++-- tests/test_rest.py | 3 ++- ui/js/src/kimchi.network.js | 5 +++++ ui/pages/i18n.json.tmpl | 1 + 6 files changed, 41 insertions(+), 3 deletions(-) -- 1.9.1

5 10

[PATCH] Supress error messages while checking vm metadata
by Ramon Medeiros 27 Apr '15

27 Apr '15

From: Ramon Medeiros <ramonn(a)jarvis.br.ibm.com> The function FeatureTests.disable_libvirt_error_logging only removes the error message that will be displayed on the console. Libvirt also logs errors on messages log. To avoid flood messages log with metadata errors, it will be verified manually if the xml has the tag. Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> --- src/kimchi/model/utils.py | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/kimchi/model/utils.py b/src/kimchi/model/utils.py index b2739b2..749be93 100644 --- a/src/kimchi/model/utils.py +++ b/src/kimchi/model/utils.py @@ -22,6 +22,7 @@ import socket import urlparse from lxml import etree, objectify from lxml.builder import E, ElementMaker +from xml.dom import minidom from kimchi.exception import OperationFailed from kimchi.model.featuretests import FeatureTests @@ -95,6 +96,13 @@ def _kimchi_set_metadata_node(dom, node): def libvirt_get_kimchi_metadata_node(dom, mode="current"): + # check xml for metadata tag + xml = minidom.parseString(dom.XMLDesc(0)) + + # no metadata: return + if not xml.getElementsByTagName('metadata'): + return None + FeatureTests.disable_libvirt_error_logging() try: xml = dom.metadata(libvirt.VIR_DOMAIN_METADATA_ELEMENT, -- 2.1.0

2 1

[PATCH v4 0/7] Async VM Creation
by Christy Perez 27 Apr '15

27 Apr '15

If a guest has a large disk, and uses a filesystem that requires preallocation, it can take several minutes to create a VM. During that time, kimchi is tied up by the VM creation. This patch changes the VMs Collection to be an AsyncCollection. Another change required for this was to create a more granular way to query tasks. Currently it is not possible (using the API) to query tasks for the same collection or resource type that may have different operations. For example, VM cloning is also an asynchronous operation. For the guests tab, the UI was querying all running tasks and displaying them with the Cloning label. This picked up VMs that were being created as well. For more information about how the tasks can be queried, see the updated API doc and the UI change. Christy Perez (7): Granular Task Queries: Backend Granular task query test updates Granular Task Queries: UI Create guests asynchronously: Backend Async vm creation test updates VMs Async Create: UI Translation update to add 'Creating' docs/API.md | 16 ++++++ po/de_DE.po | 103 ++++++++++++++++++++----------------- po/en_US.po | 83 +++++++++++++++++------------- po/es_ES.po | 102 +++++++++++++++++++----------------- po/fr_FR.po | 102 +++++++++++++++++++----------------- po/it_IT.po | 102 +++++++++++++++++++----------------- po/ja_JP.po | 102 +++++++++++++++++++----------------- po/kimchi.pot | 84 +++++++++++++++++------------- po/ko_KR.po | 102 +++++++++++++++++++----------------- po/pt_BR.po | 102 +++++++++++++++++++----------------- po/ru_RU.po | 102 +++++++++++++++++++----------------- po/zh_CN.po | 102 +++++++++++++++++++----------------- po/zh_TW.po | 102 +++++++++++++++++++----------------- src/kimchi/control/vms.py | 4 +- src/kimchi/mockmodel.py | 7 +-- src/kimchi/model/debugreports.py | 4 +- src/kimchi/model/host.py | 4 +- src/kimchi/model/storagepools.py | 2 +- src/kimchi/model/storagevolumes.py | 12 +++-- src/kimchi/model/vms.py | 33 +++++++++--- src/kimchi/model/vmsnapshots.py | 5 +- tests/test_authorization.py | 23 +++++---- tests/test_mockmodel.py | 12 +++-- tests/test_model.py | 60 ++++++++++++++------- tests/test_model_storagevolume.py | 2 +- tests/test_rest.py | 77 ++++++++++++++++++++------- ui/css/theme-default/list.css | 18 +++++++ ui/js/src/kimchi.guest_main.js | 29 +++++++++-- ui/pages/guest.html.tmpl | 3 ++ 29 files changed, 886 insertions(+), 613 deletions(-) -- 2.1.0

4 22

[PATCH V2 0/2] Warn users about NetworkManager instability
by Jose Ricardo Ziviani 27 Apr '15

27 Apr '15

V2: - Change is_nm_running to use a 'cross distro' command. - Update the warning message NetworkManager might have some instability when managing the bridges created by libvirt. There some bugs against NM about it and there is no much we can do more than warn the users about it. So, this patchset checks if NM is running and, if so, will display a message to users. Jose Ricardo Ziviani (2): Implement function to check if NM is running. Add a warn about NM running in the system. src/kimchi/model/config.py | 1 + src/kimchi/model/featuretests.py | 10 ++++++++++ tests/test_rest.py | 3 ++- ui/js/src/kimchi.network.js | 5 +++++ ui/pages/i18n.json.tmpl | 1 + 5 files changed, 19 insertions(+), 1 deletion(-) -- 1.9.1

3 6

[PATCH] Host tests
by Aline Manera 27 Apr '15

27 Apr '15

This patch creates a new test file (test_host.py) to handle all the Host Resource related tests. It also updates the docs/API.md to reflect the right data set returned for /host/partitions API. Signed-off-by: Aline Manera <alinefm(a)linux.vnet.ibm.com> --- docs/API.md | 1 + src/kimchi/mockmodel.py | 8 +- tests/test_host.py | 192 ++++++++++++++++++++++++++++++++++++++++++++++++ tests/test_mockmodel.py | 17 ----- tests/test_model.py | 73 ------------------ tests/test_rest.py | 72 ------------------ 6 files changed, 200 insertions(+), 163 deletions(-) create mode 100644 tests/test_host.py diff --git a/docs/API.md b/docs/API.md index 3f7925f..922ded1 100644 --- a/docs/API.md +++ b/docs/API.md @@ -985,6 +985,7 @@ stats history * size: The total size of the partition, in bytes * mountpoint: If the partition is mounted, represents the mountpoint. Otherwise blank. + * available: false, if the partition is in use by system; true, otherwise. ### Collection: Devices diff --git a/src/kimchi/mockmodel.py b/src/kimchi/mockmodel.py index 413ac5d..3ebddca 100644 --- a/src/kimchi/mockmodel.py +++ b/src/kimchi/mockmodel.py @@ -42,7 +42,7 @@ from kimchi.model.templates import LibvirtVMTemplate from kimchi.model.users import PAMUsersModel from kimchi.model.groups import PAMGroupsModel from kimchi.objectstore import ObjectStore -from kimchi.utils import add_task, get_next_clone_name +from kimchi.utils import add_task, get_next_clone_name, kimchi_log from kimchi.vmtemplate import VMTemplate from kimchi.xmlutils.utils import xml_item_update @@ -273,6 +273,12 @@ class MockModel(Model): conn.storagePoolDefineXML(ET.tostring(root), 0) + def _mock_host_shutdown(self, *name): + kimchi_log.info("The host system will be shutted down") + + def _mock_host_reboot(self, *name): + kimchi_log.info("The host system will be rebooted") + def _mock_storagevolumes_create(self, pool, params): vol_source = ['file', 'url', 'capacity'] index_list = list(i for i in range(len(vol_source)) diff --git a/tests/test_host.py b/tests/test_host.py new file mode 100644 index 0000000..1273457 --- /dev/null +++ b/tests/test_host.py @@ -0,0 +1,192 @@ +# -*- coding: utf-8 -*- +# +# Project Kimchi +# +# Copyright IBM, Corp. 2015 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +import json +import os +import platform +import psutil +import tempfile +import time +import unittest + +from functools import partial + +from kimchi.mockmodel import MockModel +from utils import get_free_port, patch_auth, request, run_server, wait_task + +test_server = None +model = None +host = None +ssl_port = None +tmpfile = None + + +def setUpModule(): + global test_server, model, host, ssl_port, tmpfile + + patch_auth() + tmpfile = tempfile.mktemp() + model = MockModel(tmpfile) + host = '127.0.0.1' + port = get_free_port('http') + ssl_port = get_free_port('https') + cherrypy_port = get_free_port('cherrypy_port') + test_server = run_server(host, port, ssl_port, test_mode=True, + cherrypy_port=cherrypy_port, model=model) + + +def tearDownModule(): + test_server.stop() + os.unlink(tmpfile) + + +class HostTests(unittest.TestCase): + def setUp(self): + self.request = partial(request, host, ssl_port) + + def test_hostinfo(self): + resp = self.request('/host').read() + info = json.loads(resp) + keys = ['os_distro', 'os_version', 'os_codename', 'cpu_model', + 'memory', 'cpus'] + self.assertEquals(sorted(keys), sorted(info.keys())) + + distro, version, codename = platform.linux_distribution() + self.assertEquals(distro, info['os_distro']) + self.assertEquals(version, info['os_version']) + self.assertEquals(unicode(codename, "utf-8"), info['os_codename']) + self.assertEquals(psutil.TOTAL_PHYMEM, info['memory']) + + def test_hoststats(self): + time.sleep(1) + stats_keys = ['cpu_utilization', 'memory', 'disk_read_rate', + 'disk_write_rate', 'net_recv_rate', 'net_sent_rate'] + resp = self.request('/host/stats').read() + stats = json.loads(resp) + self.assertEquals(sorted(stats_keys), sorted(stats.keys())) + + cpu_utilization = stats['cpu_utilization'] + self.assertIsInstance(cpu_utilization, float) + self.assertGreaterEqual(cpu_utilization, 0.0) + self.assertTrue(cpu_utilization <= 100.0) + + memory_stats = stats['memory'] + self.assertIn('total', memory_stats) + self.assertIn('free', memory_stats) + self.assertIn('cached', memory_stats) + self.assertIn('buffers', memory_stats) + self.assertIn('avail', memory_stats) + + resp = self.request('/host/stats/history').read() + history = json.loads(resp) + self.assertEquals(sorted(stats_keys), sorted(history.keys())) + + def test_host_actions(self): + def _task_lookup(taskid): + return json.loads(self.request('/tasks/%s' % taskid).read()) + + resp = self.request('/host/shutdown', '{}', 'POST') + self.assertEquals(200, resp.status) + resp = self.request('/host/reboot', '{}', 'POST') + self.assertEquals(200, resp.status) + + # Test system update + resp = self.request('/host/packagesupdate', None, 'GET') + pkgs = json.loads(resp.read()) + self.assertEquals(3, len(pkgs)) + + pkg_keys = ['package_name', 'repository', 'arch', 'version'] + for p in pkgs: + name = p['package_name'] + resp = self.request('/host/packagesupdate/' + name, None, 'GET') + info = json.loads(resp.read()) + self.assertEquals(sorted(pkg_keys), sorted(info.keys())) + + resp = self.request('/host/swupdate', '{}', 'POST') + task = json.loads(resp.read()) + task_params = [u'id', u'message', u'status', u'target_uri'] + self.assertEquals(sorted(task_params), sorted(task.keys())) + + resp = self.request('/tasks/' + task[u'id'], None, 'GET') + task_info = json.loads(resp.read()) + self.assertEquals(task_info['status'], 'running') + wait_task(_task_lookup, task_info['id']) + resp = self.request('/tasks/' + task[u'id'], None, 'GET') + task_info = json.loads(resp.read()) + self.assertEquals(task_info['status'], 'finished') + self.assertIn(u'All packages updated', task_info['message']) + pkgs = model.packagesupdate_get_list() + self.assertEquals(0, len(pkgs)) + + def test_host_partitions(self): + resp = self.request('/host/partitions') + self.assertEquals(200, resp.status) + partitions = json.loads(resp.read()) + + keys = ['name', 'path', 'type', 'fstype', 'size', 'mountpoint', + 'available'] + for item in partitions: + resp = self.request('/host/partitions/%s' % item['name']) + info = json.loads(resp.read()) + self.assertEquals(sorted(info.keys()), sorted(keys)) + + def test_host_devices(self): + def asset_devices_type(devices, dev_type): + for dev in devices: + self.assertEquals(dev['device_type'], dev_type) + + resp = self.request('/host/devices?_cap=scsi_host') + nodedevs = json.loads(resp.read()) + # Mockmodel brings 3 preconfigured scsi fc_host + self.assertEquals(3, len(nodedevs)) + + nodedev = json.loads(self.request('/host/devices/scsi_host2').read()) + # Mockmodel generates random wwpn and wwnn + self.assertEquals('scsi_host2', nodedev['name']) + self.assertEquals('fc_host', nodedev['adapter']['type']) + self.assertEquals(16, len(nodedev['adapter']['wwpn'])) + self.assertEquals(16, len(nodedev['adapter']['wwnn'])) + + devs = json.loads(self.request('/host/devices').read()) + dev_names = [dev['name'] for dev in devs] + for dev_type in ('pci', 'usb_device', 'scsi'): + resp = self.request('/host/devices?_cap=%s' % dev_type) + devsByType = json.loads(resp.read()) + names = [dev['name'] for dev in devsByType] + self.assertTrue(set(names) <= set(dev_names)) + asset_devices_type(devsByType, dev_type) + + resp = self.request('/host/devices?_passthrough=true') + passthru_devs = [dev['name'] for dev in json.loads(resp.read())] + self.assertTrue(set(passthru_devs) <= set(dev_names)) + + for dev_type in ('pci', 'usb_device', 'scsi'): + resp = self.request('/host/devices?_cap=%s&_passthrough=true' % + dev_type) + filteredDevs = json.loads(resp.read()) + filteredNames = [dev['name'] for dev in filteredDevs] + self.assertTrue(set(filteredNames) <= set(dev_names)) + asset_devices_type(filteredDevs, dev_type) + + for dev in passthru_devs: + resp = self.request('/host/devices?_passthrough_affected_by=%s' % + dev) + affected_devs = [dev['name'] for dev in json.loads(resp.read())] + self.assertTrue(set(affected_devs) <= set(dev_names)) diff --git a/tests/test_mockmodel.py b/tests/test_mockmodel.py index c7e733e..aa48dd1 100644 --- a/tests/test_mockmodel.py +++ b/tests/test_mockmodel.py @@ -26,7 +26,6 @@ import unittest import kimchi.mockmodel from utils import get_free_port, patch_auth, request, run_server -from utils import wait_task from kimchi.osinfo import get_template_default @@ -131,19 +130,3 @@ class MockModelTests(unittest.TestCase): self.assertEquals(stats_keys, set(info['stats'].keys())) self.assertEquals('vnc', info['graphics']['type']) self.assertEquals('127.0.0.1', info['graphics']['listen']) - - def test_packages_update(self): - pkgs = model.packagesupdate_get_list() - self.assertEquals(3, len(pkgs)) - - for pkg_name in pkgs: - pkgupdate = model.packageupdate_lookup(pkg_name) - self.assertIn('package_name', pkgupdate.keys()) - self.assertIn('repository', pkgupdate.keys()) - self.assertIn('arch', pkgupdate.keys()) - self.assertIn('version', pkgupdate.keys()) - - task = model.host_swupdate() - task_params = [u'id', u'message', u'status', u'target_uri'] - self.assertEquals(sorted(task_params), sorted(task.keys())) - wait_task(model.task_lookup, task['id']) diff --git a/tests/test_model.py b/tests/test_model.py index c25929f..210adfd 100644 --- a/tests/test_model.py +++ b/tests/test_model.py @@ -20,8 +20,6 @@ import grp import os -import platform -import psutil import pwd import re import shutil @@ -725,37 +723,6 @@ class ModelTests(unittest.TestCase): self.assertIn('ipaddr', iface) self.assertIn('netmask', iface) - @unittest.skipUnless(utils.running_as_root(), 'Must be run as root') - def test_get_devices(self): - def asset_devices_type(devices, dev_type): - for dev in devices: - self.assertEquals(dev['device_type'], dev_type) - - inst = model.Model('qemu:///system', - objstore_loc=self.tmp_store) - - devs = inst.devices_get_list() - - for dev_type in ('pci', 'usb_device', 'scsi'): - names = inst.devices_get_list(_cap=dev_type) - self.assertTrue(set(names) <= set(devs)) - infos = [inst.device_lookup(name) for name in names] - asset_devices_type(infos, dev_type) - - passthru_devs = inst.devices_get_list(_passthrough='true') - self.assertTrue(set(passthru_devs) <= set(devs)) - - for dev_type in ('pci', 'usb_device', 'scsi'): - names = inst.devices_get_list(_cap=dev_type, _passthrough='true') - self.assertTrue(set(names) <= set(devs)) - infos = [inst.device_lookup(name) for name in names] - asset_devices_type(infos, dev_type) - - for dev_name in passthru_devs: - affected_devs = inst.devices_get_list( - _passthrough_affected_by=dev_name) - self.assertTrue(set(affected_devs) <= set(devs)) - def test_async_tasks(self): class task_except(Exception): pass @@ -983,46 +950,6 @@ class ModelTests(unittest.TestCase): self.assertIn('path', distro) @unittest.skipUnless(utils.running_as_root(), 'Must be run as root') - def test_get_hostinfo(self): - inst = model.Model(None, - objstore_loc=self.tmp_store) - info = inst.host_lookup() - distro, version, codename = platform.linux_distribution() - self.assertIn('cpu_model', info) - self.assertIn('cpus', info) - self.assertEquals(distro, info['os_distro']) - self.assertEquals(version, info['os_version']) - self.assertEquals(unicode(codename, "utf-8"), info['os_codename']) - self.assertEquals(psutil.TOTAL_PHYMEM, info['memory']) - - def test_get_hoststats(self): - inst = model.Model('test:///default', - objstore_loc=self.tmp_store) - time.sleep(1.5) - stats = inst.hoststats_lookup() - stats_keys = ['cpu_utilization', 'memory', 'disk_read_rate', - 'disk_write_rate', 'net_recv_rate', 'net_sent_rate'] - self.assertEquals(sorted(stats_keys), sorted(stats.keys())) - cpu_utilization = stats['cpu_utilization'] - # cpu_utilization is set int 0, after first stats sample - # the cpu_utilization is float in range [0.0, 100.0] - self.assertIsInstance(cpu_utilization, float) - self.assertGreaterEqual(cpu_utilization, 0.0) - self.assertTrue(cpu_utilization <= 100.0) - - memory_stats = stats['memory'] - self.assertIn('total', memory_stats) - self.assertIn('free', memory_stats) - self.assertIn('cached', memory_stats) - self.assertIn('buffers', memory_stats) - self.assertIn('avail', memory_stats) - - history = inst.hoststatshistory_lookup() - self.assertEquals(sorted(stats_keys), sorted(history.keys())) - for key, value in history.iteritems(): - self.assertEquals(type(value), list) - - @unittest.skipUnless(utils.running_as_root(), 'Must be run as root') def test_deep_scan(self): inst = model.Model(None, objstore_loc=self.tmp_store) diff --git a/tests/test_rest.py b/tests/test_rest.py index ee350b2..4ade722 100644 --- a/tests/test_rest.py +++ b/tests/test_rest.py @@ -93,19 +93,6 @@ class RestTests(unittest.TestCase): resp = self.request(*args) self.assertEquals(code, resp.status) - def test_host_devices(self): - resp = self.request('/host/devices?_cap=scsi_host') - nodedevs = json.loads(resp.read()) - # Mockmodel brings 3 preconfigured scsi fc_host - self.assertEquals(3, len(nodedevs)) - - nodedev = json.loads(self.request('/host/devices/scsi_host2').read()) - # Mockmodel generates random wwpn and wwnn - self.assertEquals('scsi_host2', nodedev['name']) - self.assertEquals('fc_host', nodedev['adapter']['type']) - self.assertEquals(16, len(nodedev['adapter']['wwpn'])) - self.assertEquals(16, len(nodedev['adapter']['wwnn'])) - def test_get_vms(self): vms = json.loads(self.request('/vms').read()) # test_rest.py uses MockModel() which connects to libvirt URI @@ -1129,65 +1116,6 @@ class RestTests(unittest.TestCase): resp = request(host, ssl_port, debugre['uri']) self.assertEquals(200, resp.status) - def test_host(self): - resp = self.request('/host').read() - info = json.loads(resp) - - keys = ['os_distro', 'os_version', 'os_codename', 'cpu_model', - 'memory', 'cpus'] - self.assertEquals(sorted(keys), sorted(info.keys())) - - def test_hoststats(self): - stats_keys = ['cpu_utilization', 'memory', 'disk_read_rate', - 'disk_write_rate', 'net_recv_rate', 'net_sent_rate'] - resp = self.request('/host/stats').read() - stats = json.loads(resp) - self.assertEquals(sorted(stats_keys), sorted(stats.keys())) - - cpu_utilization = stats['cpu_utilization'] - self.assertIsInstance(cpu_utilization, float) - self.assertGreaterEqual(cpu_utilization, 0.0) - self.assertTrue(cpu_utilization <= 100.0) - - memory_stats = stats['memory'] - self.assertIn('total', memory_stats) - self.assertIn('free', memory_stats) - self.assertIn('cached', memory_stats) - self.assertIn('buffers', memory_stats) - self.assertIn('avail', memory_stats) - - resp = self.request('/host/stats/history').read() - history = json.loads(resp) - self.assertEquals(sorted(stats_keys), sorted(history.keys())) - - def test_packages_update(self): - resp = self.request('/host/packagesupdate', None, 'GET') - pkgs = json.loads(resp.read()) - self.assertEquals(3, len(pkgs)) - - for p in pkgs: - name = p['package_name'] - resp = self.request('/host/packagesupdate/' + name, None, 'GET') - info = json.loads(resp.read()) - self.assertIn('package_name', info.keys()) - self.assertIn('repository', info.keys()) - self.assertIn('arch', info.keys()) - self.assertIn('version', info.keys()) - - resp = self.request('/host/swupdate', '{}', 'POST') - task = json.loads(resp.read()) - task_params = [u'id', u'message', u'status', u'target_uri'] - self.assertEquals(sorted(task_params), sorted(task.keys())) - - resp = self.request('/tasks/' + task[u'id'], None, 'GET') - task_info = json.loads(resp.read()) - self.assertEquals(task_info['status'], 'running') - wait_task(self._task_lookup, task_info['id']) - resp = self.request('/tasks/' + task[u'id'], None, 'GET') - task_info = json.loads(resp.read()) - self.assertEquals(task_info['status'], 'finished') - self.assertIn(u'All packages updated', task_info['message']) - def test_repositories(self): def verify_repo(t, res): for field in ('repo_id', 'enabled', 'baseurl', 'config'): -- 2.1.0

2 2

[PATCH] Move kimchi nginx config file to nginx default directory.
by Jose Ricardo Ziviani 23 Apr '15

23 Apr '15

- Move Kimchi nginx config file to /etc/nginx/conf.d. - Rename nginx_kimchi.conf to kimchi.conf. Signed-off-by: Jose Ricardo Ziviani <joserz(a)linux.vnet.ibm.com> --- .gitignore | 2 +- configure.ac | 1 + contrib/kimchi.spec.fedora.in | 6 ++-- contrib/kimchi.spec.suse.in | 7 ++-- src/Makefile.am | 5 ++- src/kimchi/config.py.in | 2 ++ src/kimchi/proxy.py | 12 +++---- src/nginx.conf.in | 76 ------------------------------------------- src/nginx/Makefile.am | 23 +++++++++++++ src/nginx/kimchi.conf.in | 76 +++++++++++++++++++++++++++++++++++++++++++ 10 files changed, 119 insertions(+), 91 deletions(-) delete mode 100644 src/nginx.conf.in create mode 100644 src/nginx/Makefile.am create mode 100644 src/nginx/kimchi.conf.in diff --git a/.gitignore b/.gitignore index 4abfc0a..a318bd9 100644 --- a/.gitignore +++ b/.gitignore @@ -28,7 +28,7 @@ stamp-po kimchi-*.tar.gz src/kimchid src/kimchi.conf -src/nginx_kimchi.conf +src/nginx/kimchi.conf src/kimchi/config.py tests/run_tests.sh tests/test_config.py diff --git a/configure.ac b/configure.ac index 1b476c3..5da3240 100644 --- a/configure.ac +++ b/configure.ac @@ -80,6 +80,7 @@ AC_CONFIG_FILES([ docs/Makefile src/Makefile src/distros.d/Makefile + src/nginx/Makefile src/kimchi/Makefile src/kimchi/control/Makefile src/kimchi/control/vm/Makefile diff --git a/contrib/kimchi.spec.fedora.in b/contrib/kimchi.spec.fedora.in index 750dada..a721a7f 100644 --- a/contrib/kimchi.spec.fedora.in +++ b/contrib/kimchi.spec.fedora.in @@ -94,7 +94,7 @@ touch %{buildroot}/%{_localstatedir}/log/kimchi/kimchi-error.log # create /etc/kimchi structure mkdir -p %{buildroot}/%{_sysconfdir}/kimchi/ -touch %{buildroot}/%{_sysconfdir}/kimchi/nginx_kimchi.conf +touch %{buildroot}/%{_sysconfdir}/nginx/conf.d/kimchi.conf # Install the systemd scripts install -Dm 0644 contrib/kimchid.service.fedora %{buildroot}%{_unitdir}/kimchid.service @@ -158,14 +158,14 @@ rm -rf $RPM_BUILD_ROOT %{_datadir}/kimchi/config/ui/*.xml %{_datadir}/kimchi/ui/ %{_datadir}/kimchi +%{_sysconfdir}/nginx/conf.d/kimchi.conf.in +%{_sysconfdir}/nginx/conf.d/kimchi.conf %{_sysconfdir}/kimchi/kimchi.conf -%{_sysconfdir}/kimchi/nginx.conf.in %{_sysconfdir}/kimchi/distros.d/debian.json %{_sysconfdir}/kimchi/distros.d/fedora.json %{_sysconfdir}/kimchi/distros.d/opensuse.json %{_sysconfdir}/kimchi/distros.d/ubuntu.json %{_sysconfdir}/kimchi/distros.d/gentoo.json -%{_sysconfdir}/kimchi/nginx_kimchi.conf %{_sysconfdir}/kimchi/ %{_sharedstatedir}/kimchi/debugreports/ %{_sharedstatedir}/kimchi/screenshots/ diff --git a/contrib/kimchi.spec.suse.in b/contrib/kimchi.spec.suse.in index 7e4172d..5673ced 100644 --- a/contrib/kimchi.spec.suse.in +++ b/contrib/kimchi.spec.suse.in @@ -63,7 +63,9 @@ touch %{buildroot}/%{_localstatedir}/log/kimchi/kimchi-error.log # create /etc/kimchi structure mkdir -p %{buildroot}/%{_sysconfdir}/kimchi/ -touch %{buildroot}/%{_sysconfdir}/kimchi/nginx_kimchi.conf + +# create nginx configuration +touch %{buildroot}/%{_sysconfdir}/nginx/conf.d/kimchi.conf # Install the SysV init scripts install -Dm 0755 contrib/kimchid.sysvinit %{buildroot}%{_initrddir}/kimchid @@ -97,13 +99,14 @@ rm -rf $RPM_BUILD_ROOT %{_datadir}/kimchi/config/ui/*.xml %{_datadir}/kimchi/ui/ %{_datadir}/kimchi +%{_sysconfdir}/nginx/conf.d/kimchi.conf.in +%{_sysconfdir}/nginx/conf.d/kimchi.conf %{_sysconfdir}/kimchi/kimchi.conf %{_sysconfdir}/kimchi/distros.d/debian.json %{_sysconfdir}/kimchi/distros.d/fedora.json %{_sysconfdir}/kimchi/distros.d/opensuse.json %{_sysconfdir}/kimchi/distros.d/ubuntu.json %{_sysconfdir}/kimchi/distros.d/gentoo.json -%{_sysconfdir}/kimchi/nginx_kimchi.conf %{_sysconfdir}/kimchi %{_initrddir}/kimchid %{_sysconfdir}/kimchi/ diff --git a/src/Makefile.am b/src/Makefile.am index dfeb24e..edc163e 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -17,18 +17,17 @@ # License along with this library; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA -SUBDIRS = kimchi distros.d +SUBDIRS = kimchi distros.d nginx EXTRA_DIST = kimchid.in \ kimchi.conf.in \ - nginx.conf.in \ firewalld.xml \ $(NULL) bin_SCRIPTS = kimchid confdir = $(sysconfdir)/kimchi -dist_conf_DATA = kimchi.conf nginx.conf.in +dist_conf_DATA = kimchi.conf BUILT_SOURCES = kimchi.conf diff --git a/src/kimchi/config.py.in b/src/kimchi/config.py.in index f2e1cac..f15d228 100644 --- a/src/kimchi/config.py.in +++ b/src/kimchi/config.py.in @@ -112,6 +112,7 @@ class Paths(object): self.novnc_dir = '/usr/share/novnc' if self.installed: + self.nginx_conf_dir = '@sysconfdir@/nginx/conf.d' self.state_dir = '@localstatedir@/lib/kimchi' self.log_dir = '@localstatedir@/log/kimchi' self.conf_dir = '@sysconfdir@/kimchi' @@ -120,6 +121,7 @@ class Paths(object): self.mo_dir = '@prefix@/share/locale' self.spice_css_file = os.path.join(self.spice_dir, 'spice.css') else: + self.nginx_conf_dir = self.add_prefix('src/nginx') self.state_dir = self.add_prefix('data') self.log_dir = self.add_prefix('log') self.conf_dir = self.add_prefix('src') diff --git a/src/kimchi/proxy.py b/src/kimchi/proxy.py index fafa5bc..1b70d5e 100644 --- a/src/kimchi/proxy.py +++ b/src/kimchi/proxy.py @@ -37,8 +37,7 @@ def _create_proxy_config(options): To allow flexibility in which port kimchi runs, we need the same flexibility with the nginx proxy. This method creates the config file dynamically by using 'nginx.conf.in' as a template, creating - the file 'nginx_kimchi.config' which will be used to launch the - proxy. + the file 'kimchi.conf' which will be used to launch the proxy. Arguments: options - OptionParser object with Kimchi config options @@ -53,6 +52,7 @@ def _create_proxy_config(options): user_proxy = 'www-data' config_dir = paths.conf_dir + nginx_config_dir = paths.nginx_conf_dir cert = options.ssl_cert key = options.ssl_key @@ -70,7 +70,7 @@ def _create_proxy_config(options): # Read template file and create a new config file # with the specified parameters. - with open(os.path.join(config_dir, "nginx.conf.in")) as template: + with open(os.path.join(nginx_config_dir, "kimchi.conf.in")) as template: data = template.read() data = Template(data) data = data.safe_substitute(user=user_proxy, @@ -81,7 +81,7 @@ def _create_proxy_config(options): max_body_size=eval(options.max_body_size)) # Write file to be used for nginx. - config_file = open(os.path.join(config_dir, "nginx_kimchi.conf"), "w") + config_file = open(os.path.join(nginx_config_dir, "kimchi.conf"), "w") config_file.write(data) config_file.close() @@ -89,8 +89,8 @@ def _create_proxy_config(options): def start_proxy(options): """Start nginx reverse proxy.""" _create_proxy_config(options) - config_dir = paths.conf_dir - config_file = "%s/nginx_kimchi.conf" % config_dir + nginx_config_dir = paths.nginx_conf_dir + config_file = "%s/kimchi.conf" % nginx_config_dir cmd = ['nginx', '-c', config_file] subprocess.call(cmd) diff --git a/src/nginx.conf.in b/src/nginx.conf.in deleted file mode 100644 index e308152..0000000 --- a/src/nginx.conf.in +++ /dev/null @@ -1,76 +0,0 @@ -# Project Kimchi -# -# Copyright IBM, Corp. 2014 -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library; if not, write to the Free Software -# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA -# 02110-1301 USA - -# This is a template file to be used to generate a nginx -# proxy config file at kimchid script. - -user ${user}; -worker_processes 1; - -error_log /var/log/nginx/error.log; - -events { - worker_connections 1024; -} - -http { - - log_format main '$remote_addr - $remote_user [$time_local] "$request" ' - '$status $body_bytes_sent "$http_referer" ' - '"$http_user_agent" "$http_x_forwarded_for"'; - - access_log /var/log/nginx/access.log main; - sendfile on; - - client_max_body_size ${max_body_size}k; - - # Timeout set to 10 minutes to avoid the 504 Gateway Timeout - # when Kimchi is processing a request. - proxy_connect_timeout 600; - proxy_send_timeout 600; - proxy_read_timeout 600; - send_timeout 600; - - server { - listen ${proxy_ssl_port} ssl; - - ssl_certificate ${cert_pem}; - ssl_certificate_key ${cert_key}; - ssl_protocols TLSv1 TLSv1.1 TLSv1.2; - ssl_ciphers ECDH@STRENGTH:DH@STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNULL; - - add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;"; - add_header X-Frame-Options DENY; - add_header X-Content-Type-Options nosniff; - add_header X-XSS-Protection "1; mode=block"; - - location / { - proxy_pass http://127.0.0.1:${kimchid_port}; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_redirect http://127.0.0.1:${kimchid_port}/ https://$host:${proxy_ssl_port}/; - } - } - - server { - listen ${proxy_port}; - rewrite ^/(.*)$ https://$host:${proxy_ssl_port}/$1 redirect; - } -} diff --git a/src/nginx/Makefile.am b/src/nginx/Makefile.am new file mode 100644 index 0000000..b240508 --- /dev/null +++ b/src/nginx/Makefile.am @@ -0,0 +1,23 @@ +# +# Kimchi +# +# Copyright IBM Corp, 2013 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +EXTRA_DIST = kimchi.conf.in + +confdir = $(sysconfdir)/nginx/conf.d +dist_conf_DATA = kimchi.conf.in diff --git a/src/nginx/kimchi.conf.in b/src/nginx/kimchi.conf.in new file mode 100644 index 0000000..e308152 --- /dev/null +++ b/src/nginx/kimchi.conf.in @@ -0,0 +1,76 @@ +# Project Kimchi +# +# Copyright IBM, Corp. 2014 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA +# 02110-1301 USA + +# This is a template file to be used to generate a nginx +# proxy config file at kimchid script. + +user ${user}; +worker_processes 1; + +error_log /var/log/nginx/error.log; + +events { + worker_connections 1024; +} + +http { + + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; + + access_log /var/log/nginx/access.log main; + sendfile on; + + client_max_body_size ${max_body_size}k; + + # Timeout set to 10 minutes to avoid the 504 Gateway Timeout + # when Kimchi is processing a request. + proxy_connect_timeout 600; + proxy_send_timeout 600; + proxy_read_timeout 600; + send_timeout 600; + + server { + listen ${proxy_ssl_port} ssl; + + ssl_certificate ${cert_pem}; + ssl_certificate_key ${cert_key}; + ssl_protocols TLSv1 TLSv1.1 TLSv1.2; + ssl_ciphers ECDH@STRENGTH:DH@STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNULL; + + add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;"; + add_header X-Frame-Options DENY; + add_header X-Content-Type-Options nosniff; + add_header X-XSS-Protection "1; mode=block"; + + location / { + proxy_pass http://127.0.0.1:${kimchid_port}; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_redirect http://127.0.0.1:${kimchid_port}/ https://$host:${proxy_ssl_port}/; + } + } + + server { + listen ${proxy_port}; + rewrite ^/(.*)$ https://$host:${proxy_ssl_port}/$1 redirect; + } +} -- 1.9.1

1 0

[kimchi-devel][PATCH 0/6] Pool on existent VG
by lvroyce＠linux.vnet.ibm.com 23 Apr '15

23 Apr '15

From: Royce Lv <lvroyce(a)linux.vnet.ibm.com> Royce Lv (6): Storagepool on existent VG: Update document Add helper function to retrieve vg information Update controller Update model for existent VG Update mockmodel tests Fix fh leak in run_command subprocess docs/API.md | 20 ++++++++++++++++ src/kimchi/control/host.py | 6 +++++ src/kimchi/i18n.py | 2 ++ src/kimchi/mockmodel.py | 9 ++++++++ src/kimchi/model/libvirtstoragepool.py | 11 +++++---- src/kimchi/model/storagepools.py | 28 +++++++++++++---------- src/kimchi/model/vgs.py | 42 ++++++++++++++++++++++++++++++++++ src/kimchi/utils.py | 22 ++++++++++++++++-- tests/test_mock_storagepool.py | 9 +++++++- 9 files changed, 130 insertions(+), 19 deletions(-) create mode 100644 src/kimchi/model/vgs.py -- 2.1.0

2 7

[RFC] Guest Network Boot support [Part1] - Create template and guest with support to Direct Boot
by Rodrigo Trujillo 20 Apr '15

20 Apr '15

Alan Evangelista (https://github.com/alanoe) and I are willing to implement the support to Network Boot to Kimchi guests. This is the first approach [Part1] we are going to develop: The direct boot When finished, we are going to send [Part2] that will cover DHCP/TFTP, etc. ------- This feature aims to develop Kimchi's support to create Guest that will boot over Network using Libvirt Direct Boot Libvirt details: https://libvirt.org/formatdomain.html#elementsOSKernel Kimchi related bug: https://github.com/kimchi-project/kimchi/issues/372 We see three major changes in Kimchi code: 1- Change Template creation and update (BACKEND) 2- Change Template creation and update (FRONTEND) 3- Change Guest creation (BACKEND) Details: 1- Change Template creation and update (BACKEND): Backend is going to wait for these new parameters (json) in Template creation or update: "netboot": { "kernel": "<KERNEL URL>", "initrd": "<INITRD URL>", "kernelargs": "<COMMAND LINE FOR KERNEL>", "repository": "<PACKAGE REPOSITORY>" (optional) } Steps: src/kimchi/API.json -> change "templates_create" and "template_update" to receive above parameters; src/kimchi/model/templates.py -> change "TemplatesModel.create()" to check if "netboot" comes to save data properly -> change "TemplateModel.update()" to check if "netboot" comes and update data properly; doc/API.md -> change to reflect new parameters; src/kimchi/i18n.py -> add new errors if needed; Other: - Change templates auto tests if necessary; - Change "VMTemplate" class in "src/kimchi/vmtemplate.py" in order to probe DISTRO and VERSION; 2- Change Template creation and update (FRONTEND): Front-end is going to have following changes: - Add new entry in 'Add Template' window: "Network Direct Installation"; - Create new window to allow user to provide new four parameters; - Change Template update window. Add new tab to change or add network installation parameters; 3- Change Guest creation (BACKEND) Here it is necessary to change the XML creation, so we need to: src/kimchi/vmtemplate.py -> change "VMTemplate.to_vm_xml()" to check existence of parameter "netboot", then change <os> element properly. Other: - Change guests auto tests if necessary; - Change xml (VMTemplate) auto test if necessary; Rodrigo Trujillo

3 3

[PATCH v2 0/4] issue #548: Hotplug network interfaces
by Crístian Deives 20 Apr '15

20 Apr '15

Here is the difference between this and the previous patchset: - Patches 1, 2 and 3 have been added; they're just bug fixes, not exactly related to the issue #548. Patch 4 remains the same. Crístian Deives (4): Parse osinfo.lookup return parameters correctly Handle missing parameter "network" when attaching a NIC Use default network model when attaching a NIC issue #548: Hotplug network interfaces src/kimchi/i18n.py | 1 - src/kimchi/model/vmifaces.py | 57 +++++++++++++++++--------- src/kimchi/xmlutils/interface.py | 15 ++++++- tests/test_model.py | 86 ++++++++++++++++++++-------------------- tests/test_rest.py | 5 +++ 5 files changed, 100 insertions(+), 64 deletions(-) -- 2.1.0

2 6

[PATCH v2] Create empty files on rpm build
by Ramon Medeiros 20 Apr '15

20 Apr '15

Changes v2: Only remove nginx conm Issue #496 - System leftovers after uninstalling Kimchi A rpm does not remove files that are not created by the installation. So, all the files were provided by the package and now they can be uninstalled. Beyond this, the function that creates the proxy conf now accepts empty files to fill the configuration. Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> --- contrib/kimchi.spec.fedora.in | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/contrib/kimchi.spec.fedora.in b/contrib/kimchi.spec.fedora.in index 9f7bb59..750dada 100644 --- a/contrib/kimchi.spec.fedora.in +++ b/contrib/kimchi.spec.fedora.in @@ -92,6 +92,10 @@ mkdir -p %{buildroot}/%{_localstatedir}/log/kimchi/ touch %{buildroot}/%{_localstatedir}/log/kimchi/kimchi-access.log touch %{buildroot}/%{_localstatedir}/log/kimchi/kimchi-error.log +# create /etc/kimchi structure +mkdir -p %{buildroot}/%{_sysconfdir}/kimchi/ +touch %{buildroot}/%{_sysconfdir}/kimchi/nginx_kimchi.conf + # Install the systemd scripts install -Dm 0644 contrib/kimchid.service.fedora %{buildroot}%{_unitdir}/kimchid.service install -Dm 0640 src/firewalld.xml %{buildroot}%{_prefix}/lib/firewalld/services/kimchid.xml @@ -161,6 +165,7 @@ rm -rf $RPM_BUILD_ROOT %{_sysconfdir}/kimchi/distros.d/opensuse.json %{_sysconfdir}/kimchi/distros.d/ubuntu.json %{_sysconfdir}/kimchi/distros.d/gentoo.json +%{_sysconfdir}/kimchi/nginx_kimchi.conf %{_sysconfdir}/kimchi/ %{_sharedstatedir}/kimchi/debugreports/ %{_sharedstatedir}/kimchi/screenshots/ -- 1.8.3.1

4 6

[PATCH v3 0/2] Remove nginx configuration on rpm -e
by Ramon Medeiros 20 Apr '15

20 Apr '15

Changes v3: Extend changes to suse v2: Remove only nginx conf. Let certificates on the machine Ramon Medeiros (2): Create empty files on rpm build Remove nginx conf on suse contrib/kimchi.spec.fedora.in | 5 +++++ contrib/kimchi.spec.suse.in | 7 ++++++- 2 files changed, 11 insertions(+), 1 deletion(-) -- 1.8.3.1

2 3

[PATCH 1/2 V2] Set qcow2 as default disk format in new templates
by Rodrigo Trujillo 20 Apr '15

20 Apr '15

Users are allowed to provide disk format when creating a new Template. However this is optional. This option sets qcow2 as default disk format for every new Template created. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/kimchi/osinfo.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/kimchi/osinfo.py b/src/kimchi/osinfo.py index 3c31346..ede2541 100644 --- a/src/kimchi/osinfo.py +++ b/src/kimchi/osinfo.py @@ -33,7 +33,8 @@ SUPPORTED_ARCHS = {'x86': ('i386', 'i686', 'x86_64'), 'ppc64le': ('ppc64le')} -common_spec = {'cpus': 1, 'memory': 1024, 'disks': [{'index': 0, 'size': 10}], +common_spec = {'cpus': 1, 'memory': 1024, 'disks': [{'index': 0, 'size': 10, + 'format': 'qcow2'}], 'cdrom_bus': 'ide', 'cdrom_index': 2, 'mouse_bus': 'ps2'} -- 2.1.0

2 4

[PATCH V2] Template: Assign 'raw' to disk format if pool is 'logical' or [i]scsi
by Rodrigo Trujillo 20 Apr '15

20 Apr '15

Kimchi always creates raw disk format to new vm disks if the storagepool is logical, iscsi or scsi. But, new templates where not using this logic and was possible, through API calls, to create a template with a SCSI storagepool and disk format vmdk, for instance. This patch fixes this issue and makes necessary change in one of the tests in order to avoid an error with _get_storage_type() and deactivated storagepools. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/kimchi/model/templates.py | 4 ++-- src/kimchi/vmtemplate.py | 5 +++++ tests/test_template.py | 2 ++ 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/src/kimchi/model/templates.py b/src/kimchi/model/templates.py index 6bc8aca..8a1820d 100644 --- a/src/kimchi/model/templates.py +++ b/src/kimchi/model/templates.py @@ -94,7 +94,7 @@ class TemplatesModel(object): # Creates the template class with necessary information # Checkings will be done while creating this class, so any exception # will be raised here - t = LibvirtVMTemplate(params, scan=True) + t = LibvirtVMTemplate(params, scan=True, conn=self.conn) name = params['name'] try: with self.objstore as session: @@ -231,8 +231,8 @@ class TemplateModel(object): class LibvirtVMTemplate(VMTemplate): def __init__(self, args, scan=False, conn=None): - VMTemplate.__init__(self, args, scan) self.conn = conn + VMTemplate.__init__(self, args, scan) def _storage_validate(self): pool_uri = self.info['storagepool'] diff --git a/src/kimchi/vmtemplate.py b/src/kimchi/vmtemplate.py index 62772e5..3e4630d 100644 --- a/src/kimchi/vmtemplate.py +++ b/src/kimchi/vmtemplate.py @@ -75,6 +75,11 @@ class VMTemplate(object): args['graphics'] = graphics self.info.update(args) + # Assign right disk format to logical and [i]scsi storagepools + if self._get_storage_type() in ['logical', 'iscsi', 'scsi']: + for i, disk in enumerate(self.info['disks']): + self.info['disks'][i]['format'] = 'raw' + def _get_os_info(self, args, scan): distro = version = 'unknown' diff --git a/tests/test_template.py b/tests/test_template.py index 23a24fb..acdb7c6 100644 --- a/tests/test_template.py +++ b/tests/test_template.py @@ -329,6 +329,8 @@ class TemplateTests(unittest.TestCase): pool = {'type': 'dir', 'name': 'dir-pool', 'path': '/tmp/dir-pool'} self.request('/storagepools', json.dumps(pool), 'POST') + pool_uri = '/storagepools/%s' % pool['name'].encode('utf-8') + self.request(pool_uri + '/activate', '{}', 'POST') # Create a template using the custom network and pool t = {'name': 'test', 'cdrom': '/tmp/mock.iso', -- 2.1.0

2 2

[PATCH V2] Fix disk format lock during VM creation
by Rodrigo Trujillo 20 Apr '15

20 Apr '15

Users are able to pass the disk format (qcow, raw, etc) in disk Template information. However, Kimchi is ignoring this information and always creating QCOW2 disk images (if the storagepool is not 'LOGICAL') when it creates a VM based on a given Template. This patch fixes this problem. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/kimchi/vmtemplate.py | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/src/kimchi/vmtemplate.py b/src/kimchi/vmtemplate.py index ec477dd..62772e5 100644 --- a/src/kimchi/vmtemplate.py +++ b/src/kimchi/vmtemplate.py @@ -193,7 +193,6 @@ class VMTemplate(object): def to_volume_list(self, vm_uuid): storage_path = self._get_storage_path() - fmt = 'raw' if self._get_storage_type() in ['logical'] else 'qcow2' ret = [] for i, d in enumerate(self.info['disks']): index = d.get('index', i) @@ -201,11 +200,16 @@ class VMTemplate(object): info = {'name': volume, 'capacity': d['size'], - 'format': fmt, + 'format': d.get('format', 'qcow2'), 'path': '%s/%s' % (storage_path, volume)} + # Validate disk format + if self._get_storage_type() in ['logical'] and \ + info['format'] != 'raw': + info['format'] = 'raw' + if 'logical' == self._get_storage_type() or \ - fmt not in ['qcow2', 'raw']: + info['format'] not in ['qcow2', 'raw']: info['allocation'] = info['capacity'] else: info['allocation'] = 0 -- 2.1.0

2 2

[RFC]Puppet deployment for Kimchi
by Royce Lv 20 Apr '15

20 Apr '15

Hi, Kimchi's initial design concept is good out of box experience for users. With features added, we have some advanced feature which needs complex user set up (such as iscsi, nfs, ldap,peer discovery etc). This makes testing, concept model set up, configuration not easy for new users. To boost user experience, I would suggest to supply puppet script samples for these set ups.

2 2

[PATCH v4 0/7] Async Disk Creation
by Christy Perez 17 Apr '15

17 Apr '15

If a guest has a large disk, and uses a filesystem that requires preallocation, it can take several minutes to create a VM. During that time, kimchi is tied up by the VM creation. This patch changes the VMs Collection to be an AsyncCollection. Another change required for this was to create a more granular way to query tasks. Currently it is not possible (using the API) to query tasks for the same collection or resource type that may have different operations. For example, VM cloning is also an asynchronous operation. For the guests tab, the UI was querying all running tasks and displaying them with the Cloning label. This picked up VMs that were being created as well. For more information about how the tasks can be queried, see the updated API doc and the UI change. Christy Perez (7): Granular Task Queries: Backend Granular task query test updates Granular Task Queries: UI Create guests asynchronously: Backend Async vm creation test updates VMs Async Create: UI Translation update to add 'Creating' docs/API.md | 16 ++++++ po/de_DE.po | 103 ++++++++++++++++++++----------------- po/en_US.po | 83 +++++++++++++++++------------- po/es_ES.po | 102 +++++++++++++++++++----------------- po/fr_FR.po | 102 +++++++++++++++++++----------------- po/it_IT.po | 102 +++++++++++++++++++----------------- po/ja_JP.po | 102 +++++++++++++++++++----------------- po/kimchi.pot | 84 +++++++++++++++++------------- po/ko_KR.po | 102 +++++++++++++++++++----------------- po/pt_BR.po | 102 +++++++++++++++++++----------------- po/ru_RU.po | 102 +++++++++++++++++++----------------- po/zh_CN.po | 102 +++++++++++++++++++----------------- po/zh_TW.po | 102 +++++++++++++++++++----------------- src/kimchi/control/vms.py | 4 +- src/kimchi/mockmodel.py | 7 +-- src/kimchi/model/debugreports.py | 4 +- src/kimchi/model/host.py | 4 +- src/kimchi/model/storagepools.py | 2 +- src/kimchi/model/storagevolumes.py | 12 +++-- src/kimchi/model/vms.py | 33 +++++++++--- src/kimchi/model/vmsnapshots.py | 5 +- tests/test_authorization.py | 23 +++++---- tests/test_mockmodel.py | 12 +++-- tests/test_model.py | 60 ++++++++++++++------- tests/test_model_storagevolume.py | 2 +- tests/test_rest.py | 77 ++++++++++++++++++++------- ui/css/theme-default/list.css | 18 +++++++ ui/js/src/kimchi.guest_main.js | 29 +++++++++-- ui/pages/guest.html.tmpl | 3 ++ 29 files changed, 886 insertions(+), 613 deletions(-) -- 2.1.0

1 8

[PATCH] Template: Assign 'raw' to disk format if pool is 'logical' or [i]scsi
by Rodrigo Trujillo 17 Apr '15

17 Apr '15

Kimchi always creates raw disk format to new vm disks if the storagepool is logical, iscsi or scsi. But, new templates where not using this logic and was possible, through API calls, to create a template with a SCSI storagepool and disk format vmdk, for instance. This patch fixes this issue. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/kimchi/model/templates.py | 4 ++-- src/kimchi/vmtemplate.py | 5 +++++ 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/src/kimchi/model/templates.py b/src/kimchi/model/templates.py index 6bc8aca..8a1820d 100644 --- a/src/kimchi/model/templates.py +++ b/src/kimchi/model/templates.py @@ -94,7 +94,7 @@ class TemplatesModel(object): # Creates the template class with necessary information # Checkings will be done while creating this class, so any exception # will be raised here - t = LibvirtVMTemplate(params, scan=True) + t = LibvirtVMTemplate(params, scan=True, conn=self.conn) name = params['name'] try: with self.objstore as session: @@ -231,8 +231,8 @@ class TemplateModel(object): class LibvirtVMTemplate(VMTemplate): def __init__(self, args, scan=False, conn=None): - VMTemplate.__init__(self, args, scan) self.conn = conn + VMTemplate.__init__(self, args, scan) def _storage_validate(self): pool_uri = self.info['storagepool'] diff --git a/src/kimchi/vmtemplate.py b/src/kimchi/vmtemplate.py index 62772e5..a309357 100644 --- a/src/kimchi/vmtemplate.py +++ b/src/kimchi/vmtemplate.py @@ -75,6 +75,11 @@ class VMTemplate(object): args['graphics'] = graphics self.info.update(args) + # Assign right disk format to logical and [i]scsi storagepools + if self._get_storage_type() in ['logical', 'iscsi', 'scsi']: + for i,disk in enumerate(self.info['disks']): + self.info['disks'][i]['format'] = 'raw' + def _get_os_info(self, args, scan): distro = version = 'unknown' -- 2.1.0

1 1

[PATCH] UI-Template Edit: Enable user to change disk format
by Rodrigo Trujillo 16 Apr '15

16 Apr '15

This patch adds a new field (Disk Format) in Storage tab in Template edit window. Users will then be allowed to select any disk format supported by Libvirt and Kimchi backend. The default disk format is qcow2 (first option), so, for compatibility, if the template register in objecstore does not contain the disk format information, the new field is going to set 'qcow2' automatically. For iscsi storagepools, the new field is going to be disabled and the format is going to be set automatically as 'raw', just like the backend behaves currently, avoiding errors. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- ui/css/theme-default/template-edit.css | 8 +++++-- ui/js/src/kimchi.template_edit_main.js | 40 +++++++++++++++++++++++++++++----- ui/pages/template-edit.html.tmpl | 16 ++++++++++++++ 3 files changed, 56 insertions(+), 8 deletions(-) diff --git a/ui/css/theme-default/template-edit.css b/ui/css/theme-default/template-edit.css index 7abee7c..ed9f02c 100644 --- a/ui/css/theme-default/template-edit.css +++ b/ui/css/theme-default/template-edit.css @@ -95,7 +95,7 @@ #edit-template-tabs .template-storage-cell{ display: inline-block; - width: 230px; + width: 180px; } #edit-template-tabs .template-storage-cell label { @@ -114,7 +114,11 @@ } #form-template-storage .template-tab-body .template-storage-name { - width: 220px; + width: 170px; +} + +#form-template-storage .template-tab-body .template-storage-disk-format { + width: 160px; } #edit-template-tabs .template-tab-body input[readonly] { diff --git a/ui/js/src/kimchi.template_edit_main.js b/ui/js/src/kimchi.template_edit_main.js index 85f76cd..48ae26b 100644 --- a/ui/js/src/kimchi.template_edit_main.js +++ b/ui/js/src/kimchi.template_edit_main.js @@ -67,7 +67,7 @@ kimchi.template_edit_main = function() { $('.template-tab-body', '#form-template-storage').append(nodeStorage); var storageOptions = ''; var scsiOptions = ''; - $('select', '#form-template-storage').find('option').remove(); + $('select:first', '#form-template-storage').find('option').remove(); $.each(result, function(index, storageEntities) { if((storageEntities.state === 'active') && (storageEntities.type != 'kimchi-iso')) { if(storageEntities.type === 'iscsi' || storageEntities.type === 'scsi') { @@ -77,7 +77,7 @@ kimchi.template_edit_main = function() { var isSlected = tmpPath === thisName ? ' selected' : ''; scsiOptions += '<option' + isSlected + '>' + tmpPath + '</option>'; }); - $('select', '#form-template-storage').append(scsiOptions); + $('select:first', '#form-template-storage').append(scsiOptions); }, function() {}); } else { var isSlected = storageEntities.name === thisName ? ' selected' : ''; @@ -85,8 +85,20 @@ kimchi.template_edit_main = function() { } } }); - $('select', '#form-template-storage').append(storageOptions); - $('select', '#form-template-storage').change(function() { + $('select:first', '#form-template-storage').append(storageOptions); + + // Set disk format + $('select:last option', '#form-template-storage').each(function() { + if ($(this).text() == storageData.storageDiskFormat) { + $(this).prop('selected', true); + } + }); + + $('select:last', '#form-template-storage').change(function() { + $('.template-storage-disk-format').val($(this).val()); + }); + + $('select:first', '#form-template-storage').change(function() { var selectedItem = $(this).parent().parent(); var tempStorageNameFull = $(this).val(); var tempName = tempStorageNameFull.split('/'); @@ -99,9 +111,19 @@ kimchi.template_edit_main = function() { kimchi.getStoragePoolVolume(tempStorageName, tempName[tempName.length-1], function(info) { volSize = info.capacity / Math.pow(1024, 3); $('.template-storage-disk', selectedItem).attr('readonly', true).val(volSize); + $('select:last option', selectedItem).each(function() { + this.selected = (this.text == 'raw'); + }); + $('select:last', selectedItem).prop('disabled', true).change(); }); } else { $('.template-storage-disk', selectedItem).attr('readonly', false); + if ($('select:last', selectedItem).prop('disabled') == true) { + $('select:last option', selectedItem).each(function() { + this.selected = (this.text == 'qcow2'); + }); + $('select:last', selectedItem).prop('disabled', false).change(); + } } }); }); @@ -120,7 +142,8 @@ kimchi.template_edit_main = function() { editMode : 'hide', storageName : defaultPool, storageType : defaultType, - storageDisk : diskEntities.size + storageDisk : diskEntities.size, + storageDiskFormat : diskEntities.format ? diskEntities.format : 'qcow2' } if (diskEntities.volume) { @@ -131,6 +154,10 @@ kimchi.template_edit_main = function() { nodeData.storageDisk = volSize; addStorageItem(nodeData); $('.template-storage-disk').attr('readonly', true); + $('select:last option', '#form-template-storage').each(function() { + this.selected = (this.text == 'raw'); + }); + $('select:last', '#form-template-storage').prop('disabled', true).change(); }); } else { addStorageItem(storageNodeData); @@ -271,7 +298,8 @@ kimchi.template_edit_main = function() { origDisks[0]['volume'] && delete origDisks[0]['volume']; origDisks[0].size = Number($('.template-storage-disk', tmpItem).val()); } - data[field] = origDisks; + origDisks[0].format = $('.template-storage-disk-format', tmpItem).val(); + data[field] = origDisks; } else if (field == 'graphics') { var type = $('#form-template-general [name="' + field + '"]').val(); diff --git a/ui/pages/template-edit.html.tmpl b/ui/pages/template-edit.html.tmpl index c7832c9..e64a30f 100644 --- a/ui/pages/template-edit.html.tmpl +++ b/ui/pages/template-edit.html.tmpl @@ -104,6 +104,7 @@ <span class="template-storage-cell">$_("Storage Pool")</span> <span class="template-storage-cell">$_("Type")</span> <span class="template-storage-cell">$_("Disk(GB)")</span> + <span class="template-storage-cell">$_("Disk Format")</span> <button type="button" id="template-edit-storage-add-button" class="action-area"></button> </div> <div class="template-tab-body"> @@ -160,6 +161,21 @@ <span class="template-storage-cell"> <input class="template-storage-disk" value={storageDisk} type="text" /> </span> + <span class="template-storage-cell"> + <input class="template-storage-disk-format" value={storageDiskFormat} type="text" style="display:none" /> + <select> + <option>qcow2</option> + <option>raw</option> + <option>bochs</option> + <option>cloop</option> + <option>cow</option> + <option>dmg</option> + <option>qcow</option> + <option>qed</option> + <option>vmdk</option> + <option>vpc</option> + </select> + </span> </div> </script> <script id="template-interface-tmpl" type="text/html"> -- 2.1.0

2 2

[PATCH] Set qcow2 as default disk format in new templates
by Rodrigo Trujillo 16 Apr '15

16 Apr '15

Users are allowed to provide disk format when creating a new Template. However this is optional. This option sets qcow2 as default disk format for every new Template created. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/kimchi/osinfo.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/kimchi/osinfo.py b/src/kimchi/osinfo.py index 3c31346..ede2541 100644 --- a/src/kimchi/osinfo.py +++ b/src/kimchi/osinfo.py @@ -33,7 +33,8 @@ SUPPORTED_ARCHS = {'x86': ('i386', 'i686', 'x86_64'), 'ppc64le': ('ppc64le')} -common_spec = {'cpus': 1, 'memory': 1024, 'disks': [{'index': 0, 'size': 10}], +common_spec = {'cpus': 1, 'memory': 1024, 'disks': [{'index': 0, 'size': 10, + 'format': 'qcow2'}], 'cdrom_bus': 'ide', 'cdrom_index': 2, 'mouse_bus': 'ps2'} -- 2.1.0

2 2

[PATCH 0/2] Objectstore and server tests
by Aline Manera 16 Apr '15

16 Apr '15

On way to complete the tests refactoring, I am sending 2 patches: one related to objectstore tests and other one about server tests. Aline Manera (2): Object store tests Server tests tests/test_mockmodel.py | 35 ------ tests/test_model.py | 71 ------------ tests/test_objectstore.py | 96 ++++++++++++++++ tests/test_rest.py | 176 ----------------------------- tests/test_server.py | 274 +++++++++++++++++++++++++++++++++++++++++++--- 5 files changed, 356 insertions(+), 296 deletions(-) create mode 100644 tests/test_objectstore.py -- 2.1.0

2 4

[PATCH] Fix disk format lock during VM creation
by Rodrigo Trujillo 16 Apr '15

16 Apr '15

Users are able to pass the disk format (qcow, raw, etc) in disk Template information. However, Kimchi is ignoring this information and always creating qcow2 disk images (if the storagepool is not 'LOGICAL') when it creates a VM based on a given Template. This patch fixes this problem, using the disk format from the template, if it does not exist, uses qcow2 as default. Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> --- src/kimchi/vmtemplate.py | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/src/kimchi/vmtemplate.py b/src/kimchi/vmtemplate.py index ec477dd..de750d4 100644 --- a/src/kimchi/vmtemplate.py +++ b/src/kimchi/vmtemplate.py @@ -193,7 +193,6 @@ class VMTemplate(object): def to_volume_list(self, vm_uuid): storage_path = self._get_storage_path() - fmt = 'raw' if self._get_storage_type() in ['logical'] else 'qcow2' ret = [] for i, d in enumerate(self.info['disks']): index = d.get('index', i) @@ -201,11 +200,16 @@ class VMTemplate(object): info = {'name': volume, 'capacity': d['size'], - 'format': fmt, + 'format': d.get('format', 'qcow2'), 'path': '%s/%s' % (storage_path, volume)} + # Validate disk format + if self._get_storage_type() in ['logical'] and \ + info['format'] != 'raw': + info['format'] = 'raw' + if 'logical' == self._get_storage_type() or \ - fmt not in ['qcow2', 'raw']: + info['format'] not in ['qcow2', 'raw']: info['allocation'] = info['capacity'] else: info['allocation'] = 0 -- 2.1.0

2 2

[PATCH] issue #548: Hotplug network interfaces
by Crístian Deives 15 Apr '15

15 Apr '15

Currently, network interfaces can only be added to shutoff VMs. However, the use might want to attach an interface to a running VM as well. Allow network interfaces to be attached also while the VM is running. The related test cases have been updated to perform the same operations on both a shutoff and a running VM. Fix issue #548 (Kimchi does not support NIC hot plug) - backend only. Signed-off-by: Crístian Deives <cristiandeives(a)gmail.com> --- src/kimchi/i18n.py | 1 - src/kimchi/model/vmifaces.py | 45 ++++++++++++++--------- tests/test_model.py | 86 ++++++++++++++++++++++---------------------- 3 files changed, 73 insertions(+), 59 deletions(-) diff --git a/src/kimchi/i18n.py b/src/kimchi/i18n.py index 46990a5..d4c1cc6 100644 --- a/src/kimchi/i18n.py +++ b/src/kimchi/i18n.py @@ -118,7 +118,6 @@ messages = { "KCHVMIF0001E": _("Interface %(iface)s does not exist in virtual machine %(name)s"), "KCHVMIF0002E": _("Network %(network)s specified for virtual machine %(name)s does not exist"), - "KCHVMIF0003E": _("Do not support guest interface hot plug attachment"), "KCHVMIF0004E": _("Supported virtual machine interfaces type is only network"), "KCHVMIF0005E": _("Network name for virtual machine interface must be a string"), "KCHVMIF0006E": _("Invalid network model card specified for virtual machine interface"), diff --git a/src/kimchi/model/vmifaces.py b/src/kimchi/model/vmifaces.py index 0c10fa9..a64315d 100644 --- a/src/kimchi/model/vmifaces.py +++ b/src/kimchi/model/vmifaces.py @@ -22,7 +22,7 @@ import random import libvirt from lxml import etree, objectify -from kimchi.exception import InvalidOperation, InvalidParameter, NotFoundError +from kimchi.exception import InvalidParameter, NotFoundError from kimchi.model.config import CapabilitiesModel from kimchi.model.vms import DOM_STATE_MAP, VMModel from kimchi.xmlutils.interface import get_iface_xml @@ -48,10 +48,6 @@ class VMIfacesModel(object): raise InvalidParameter("KCHVMIF0002E", {'name': vm, 'network': params["network"]}) - dom = VMModel.get_vm(vm, self.conn) - if DOM_STATE_MAP[dom.info()[0]] != "shutoff": - raise InvalidOperation("KCHVMIF0003E") - macs = (iface.mac.get('address') for iface in self.get_vmifaces(vm, self.conn)) @@ -60,10 +56,19 @@ class VMIfacesModel(object): if params['mac'] not in macs: break + dom = VMModel.get_vm(vm, self.conn) + os_data = VMModel.vm_get_os_metadata(dom, self.caps.metadata_support) os_distro, os_version = os_data xml = get_iface_xml(params, conn.getInfo()[0], os_distro, os_version) - dom.attachDeviceFlags(xml, libvirt.VIR_DOMAIN_AFFECT_CURRENT) + + flags = 0 + if dom.isPersistent(): + flags |= libvirt.VIR_DOMAIN_AFFECT_CONFIG + if DOM_STATE_MAP[dom.info()[0]] != "shutoff": + flags |= libvirt.VIR_DOMAIN_AFFECT_LIVE + + dom.attachDeviceFlags(xml, flags) return params['mac'] @@ -118,14 +123,16 @@ class VMIfaceModel(object): dom = VMModel.get_vm(vm, self.conn) iface = self._get_vmiface(vm, mac) - if DOM_STATE_MAP[dom.info()[0]] != "shutoff": - raise InvalidOperation("KCHVMIF0003E") - if iface is None: raise NotFoundError("KCHVMIF0001E", {'name': vm, 'iface': mac}) - dom.detachDeviceFlags(etree.tostring(iface), - libvirt.VIR_DOMAIN_AFFECT_CURRENT) + flags = 0 + if dom.isPersistent(): + flags |= libvirt.VIR_DOMAIN_AFFECT_CONFIG + if DOM_STATE_MAP[dom.info()[0]] != "shutoff": + flags |= libvirt.VIR_DOMAIN_AFFECT_LIVE + + dom.detachDeviceFlags(etree.tostring(iface), flags) def update(self, vm, mac, params): dom = VMModel.get_vm(vm, self.conn) @@ -134,16 +141,22 @@ class VMIfaceModel(object): if iface is None: raise NotFoundError("KCHVMIF0001E", {'name': vm, 'iface': mac}) - # FIXME we will support to change the live VM configuration later. + flags = 0 + if dom.isPersistent(): + flags |= libvirt.VIR_DOMAIN_AFFECT_CONFIG + if DOM_STATE_MAP[dom.info()[0]] != "shutoff": + flags |= libvirt.VIR_DOMAIN_AFFECT_LIVE + if iface.attrib['type'] == 'network' and 'network' in params: iface.source.attrib['network'] = params['network'] xml = etree.tostring(iface) - dom.updateDeviceFlags(xml, flags=libvirt.VIR_DOMAIN_AFFECT_CONFIG) - # change on the persisted VM configuration only. - if 'model' in params and dom.isPersistent(): + dom.updateDeviceFlags(xml, flags=flags) + + if 'model' in params: iface.model.attrib["type"] = params['model'] xml = etree.tostring(iface) - dom.updateDeviceFlags(xml, flags=libvirt.VIR_DOMAIN_AFFECT_CONFIG) + + dom.updateDeviceFlags(xml, flags=flags) return mac diff --git a/tests/test_model.py b/tests/test_model.py index 63bd721..39e4fe6 100644 --- a/tests/test_model.py +++ b/tests/test_model.py @@ -297,9 +297,6 @@ class ModelTests(unittest.TestCase): params = {'name': 'test', 'disks': [], 'cdrom': UBUNTU_ISO} inst.templates_create(params) rollback.prependDefer(inst.template_delete, 'test') - params = {'name': 'kimchi-ifaces', 'template': '/templates/test'} - inst.vms_create(params) - rollback.prependDefer(inst.vm_delete, 'kimchi-ifaces') # Create a network net_name = 'test-network' @@ -311,45 +308,50 @@ class ModelTests(unittest.TestCase): inst.network_activate(net_name) rollback.prependDefer(inst.network_deactivate, net_name) - ifaces = inst.vmifaces_get_list('kimchi-ifaces') - self.assertEquals(1, len(ifaces)) - - iface = inst.vmiface_lookup('kimchi-ifaces', ifaces[0]) - self.assertEquals(17, len(iface['mac'])) - self.assertEquals("default", iface['network']) - self.assertIn("model", iface) - - # attach network interface to vm - iface_args = {"type": "network", - "network": "test-network", - "model": "virtio"} - mac = inst.vmifaces_create('kimchi-ifaces', iface_args) - # detach network interface from vm - rollback.prependDefer(inst.vmiface_delete, 'kimchi-ifaces', mac) - self.assertEquals(17, len(mac)) - - iface = inst.vmiface_lookup('kimchi-ifaces', mac) - self.assertEquals("network", iface["type"]) - self.assertEquals("test-network", iface['network']) - self.assertEquals("virtio", iface["model"]) - - # attach network interface to vm without providing model - iface_args = {"type": "network", - "network": "test-network"} - mac = inst.vmifaces_create('kimchi-ifaces', iface_args) - rollback.prependDefer(inst.vmiface_delete, 'kimchi-ifaces', mac) - - iface = inst.vmiface_lookup('kimchi-ifaces', mac) - self.assertEquals("network", iface["type"]) - self.assertEquals("test-network", iface['network']) - - # update vm interface - iface_args = {"network": "default", - "model": "e1000"} - inst.vmiface_update('kimchi-ifaces', mac, iface_args) - iface = inst.vmiface_lookup('kimchi-ifaces', mac) - self.assertEquals("default", iface['network']) - self.assertEquals("e1000", iface["model"]) + for vm_name in ['kimchi-ifaces', 'kimchi-ifaces-running']: + params = {'name': vm_name, 'template': '/templates/test'} + inst.vms_create(params) + rollback.prependDefer(inst.vm_delete, vm_name) + + ifaces = inst.vmifaces_get_list(vm_name) + self.assertEquals(1, len(ifaces)) + + iface = inst.vmiface_lookup(vm_name, ifaces[0]) + self.assertEquals(17, len(iface['mac'])) + self.assertEquals("default", iface['network']) + self.assertIn("model", iface) + + # attach network interface to vm + iface_args = {"type": "network", + "network": "test-network", + "model": "virtio"} + mac = inst.vmifaces_create(vm_name, iface_args) + # detach network interface from vm + rollback.prependDefer(inst.vmiface_delete, vm_name, mac) + self.assertEquals(17, len(mac)) + + iface = inst.vmiface_lookup(vm_name, mac) + self.assertEquals("network", iface["type"]) + self.assertEquals("test-network", iface['network']) + self.assertEquals("virtio", iface["model"]) + + # attach network interface to vm without providing model + iface_args = {"type": "network", + "network": "test-network"} + mac = inst.vmifaces_create(vm_name, iface_args) + rollback.prependDefer(inst.vmiface_delete, vm_name, mac) + + iface = inst.vmiface_lookup(vm_name, mac) + self.assertEquals("network", iface["type"]) + self.assertEquals("test-network", iface['network']) + + # update vm interface + iface_args = {"network": "default", + "model": "e1000"} + inst.vmiface_update(vm_name, mac, iface_args) + iface = inst.vmiface_lookup(vm_name, mac) + self.assertEquals("default", iface['network']) + self.assertEquals("e1000", iface["model"]) @unittest.skipUnless(utils.running_as_root(), 'Must be run as root') def test_vm_disk(self): -- 2.1.0

4 3

[PATCH v2 0/3] VM suspend/resume
by Crístian Deives 13 Apr '15

13 Apr '15

This is the difference between this and the previous patchset: - Use string (instead of Unicode) when comparing VM states. Crístian Viana (3): Optimize VM update function Update some VM state conditions Add support for VM suspend and resume src/kimchi/control/vms.py | 2 ++ src/kimchi/i18n.py | 4 ++++ src/kimchi/model/vms.py | 47 ++++++++++++++++++++++++++++++++++++++++++++--- tests/test_model.py | 20 ++++++++++++++++++++ tests/test_rest.py | 28 ++++++++++++++++++++++++++++ 5 files changed, 98 insertions(+), 3 deletions(-) -- 2.1.0

2 5

[PATCH 0/2] fix: Update VM stats in "lookup"
by Crístian Deives 13 Apr '15

13 Apr '15

Crístian Deives (2): Update stats when looking up one single VM Move stats-related VM functions to VMModel src/kimchi/model/vms.py | 205 +++++++++++++++++++++++------------------------- 1 file changed, 99 insertions(+), 106 deletions(-) -- 2.1.0

2 4

Re: [Kimchi-devel] Add %Used in header in Storage tab
by Aline Manera 13 Apr '15

13 Apr '15

Applied. Thanks. Regards, Aline Manera

1 0

Re: [Kimchi-devel] Add %Used in header in Storage tab
by Aline Manera 13 Apr '15

13 Apr '15

Applied. Thanks. Regards, Aline Manera

1 0

[PATCH] Add %Used in header in Storage tab
by Socorro Stoppler 13 Apr '15

13 Apr '15

From: Socorro Stoppler <socorrob(a)us.ibm.com> Socorro Stoppler (1): Add %Used in the header for storage ui/css/theme-default/storage.css | 5 +++++ ui/pages/tabs/storage.html.tmpl | 5 ++++- 2 files changed, 9 insertions(+), 1 deletion(-) -- 1.9.1

2 3

[PATCH] Create empty files on rpm build
by Ramon Medeiros 10 Apr '15

10 Apr '15

Issue #496 - System leftovers after uninstalling Kimchi A rpm does not remove files that are not created by the installation. So, all the files were provided by the package and now they can be uninstalled. Beyond this, the function that creates the proxy conf now accepts empty files to fill the configuration. Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> --- contrib/kimchi.spec.fedora.in | 9 +++++++++ src/kimchi/proxy.py | 4 +++- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/contrib/kimchi.spec.fedora.in b/contrib/kimchi.spec.fedora.in index 9f7bb59..d06c771 100644 --- a/contrib/kimchi.spec.fedora.in +++ b/contrib/kimchi.spec.fedora.in @@ -92,6 +92,12 @@ mkdir -p %{buildroot}/%{_localstatedir}/log/kimchi/ touch %{buildroot}/%{_localstatedir}/log/kimchi/kimchi-access.log touch %{buildroot}/%{_localstatedir}/log/kimchi/kimchi-error.log +# create /etc/kimchi structure +mkdir -p %{buildroot}/%{_sysconfdir}/kimchi/ +touch %{buildroot}/%{_sysconfdir}/kimchi/kimchi-cert.pem +touch %{buildroot}/%{_sysconfdir}/kimchi/kimchi-key.pem +touch %{buildroot}/%{_sysconfdir}/kimchi/nginx_kimchi.conf + # Install the systemd scripts install -Dm 0644 contrib/kimchid.service.fedora %{buildroot}%{_unitdir}/kimchid.service install -Dm 0640 src/firewalld.xml %{buildroot}%{_prefix}/lib/firewalld/services/kimchid.xml @@ -161,6 +167,9 @@ rm -rf $RPM_BUILD_ROOT %{_sysconfdir}/kimchi/distros.d/opensuse.json %{_sysconfdir}/kimchi/distros.d/ubuntu.json %{_sysconfdir}/kimchi/distros.d/gentoo.json +%{_sysconfdir}/kimchi/kimchi-cert.pem +%{_sysconfdir}/kimchi/kimchi-key.pem +%{_sysconfdir}/kimchi/nginx_kimchi.conf %{_sysconfdir}/kimchi/ %{_sharedstatedir}/kimchi/debugreports/ %{_sharedstatedir}/kimchi/screenshots/ diff --git a/src/kimchi/proxy.py b/src/kimchi/proxy.py index fafa5bc..ecf351c 100644 --- a/src/kimchi/proxy.py +++ b/src/kimchi/proxy.py @@ -60,8 +60,10 @@ def _create_proxy_config(options): if not cert or not key: cert = '%s/kimchi-cert.pem' % config_dir key = '%s/kimchi-key.pem' % config_dir + # create cert files if they don't exist - if not os.path.exists(cert) or not os.path.exists(key): + if not (os.path.exists(cert) and len(open(cert).read()) > 0) or \ + not (os.path.exists(key) and len(open(key).read()) > 0): ssl_gen = sslcert.SSLCert() with open(cert, "w") as f: f.write(ssl_gen.cert_pem()) -- 1.8.3.1

1 0

[PATCH] Security: Prevent Bar Mitzvah attacks by disabling RC4
by Aline Manera 07 Apr '15

07 Apr '15

For reference: - http://www.theregister.co.uk/2015/03/27/bar_mitzvah_crypto_attack/ Apparently, Kimchi does not support RC4. $ curl -u <user:password> -H "Content-Type: application/json" -H "Accept: application/json" https://localhost:8001/host --ciphers RC4 --tlsv1.1 curl: (35) error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure But as we've already started some discussion to replace nginx to other reverse proxy, I prefer to have this configuration explicitily done to make us aware about that issue in future. Signed-off-by: Aline Manera <alinefm(a)linux.vnet.ibm.com> --- src/nginx.conf.in | 1 + 1 file changed, 1 insertion(+) diff --git a/src/nginx.conf.in b/src/nginx.conf.in index ed926a3..e308152 100644 --- a/src/nginx.conf.in +++ b/src/nginx.conf.in @@ -53,6 +53,7 @@ http { ssl_certificate ${cert_pem}; ssl_certificate_key ${cert_key}; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; + ssl_ciphers ECDH@STRENGTH:DH@STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNULL; add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;"; add_header X-Frame-Options DENY; -- 2.1.0

2 2

[kimchi-devel][RFC][PATCH] Storagepool on existent VG: Update document
by lvroyce＠linux.vnet.ibm.com 06 Apr '15

06 Apr '15

From: Royce Lv <lvroyce(a)linux.vnet.ibm.com> Add interfaces to report existent volume group on host. the returning detail will include what pool this volume group belongs to. If this VG is not used by any storage pool, its storage_pool will be empty string. So the process of creating pool on existent VG will be: 1. List VG not occupied by storage pool: GET /host/vgs?storage_pool='' This will retrieve volume group path and user can choose from one. 2. Create storage pool from chosen VG: POST /storagepools/ {'path':volume_group_path, 'name': storagepool_name, 'type': 'logical', 'devices':[]} Signed-off-by: Royce Lv <lvroyce(a)linux.vnet.ibm.com> --- docs/API.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/docs/API.md b/docs/API.md index 3f7925f..ed53d94 100644 --- a/docs/API.md +++ b/docs/API.md @@ -986,6 +986,26 @@ stats history * mountpoint: If the partition is mounted, represents the mountpoint. Otherwise blank. +### Collection: Volume groups + +**URI:** /host/vgs + +**Methods:** + +* **GET**: Retrieves list of volume groups. + +### Resource: Volume group + +**URI:** /host/vgs/*:name* + +**Methods:** + +* **GET**: Retrieve information of a single volume group. + * storage_pool: logical storage pool name which based on this volume group. + * path: path of this storage volume group. + * size: Total size of volume group. + * name: Name of the volume group. + ### Collection: Devices **URI:** /host/devices -- 2.1.0

2 1

[RFC] Do not block operations due to ref_cnt
by Crístian Viana 03 Apr '15

03 Apr '15

Hi, I'd like to propose a change to how Kimchi uses the resource field "ref_cnt". Currently, "ref_cnt" - which stands for "reference count" - is one of the fields returned when looking up a storage volume. Its purpose is to indicate how many times that resource is being used at the moment. For example, if the resource /storagepools/pool/storagevolumes/vol has ref_cnt=1, it means that the disk is attached to 1 VM right now and thus it cannot be attached to another VM. I believe the original idea of this feature is to prevent the same resource from being attached more than once at the same time. However, IMO, that might not always be the desired behavior and there's no way to enforce it completely as those resources can be used outside of Kimchi, where "ref_cnt" doesn't exist. For example, if I have one VM which uses the disk "vol", I'm not able to attach it to another VM via Kimchi; but if I use another libvirt-based VM manager (e.g. virsh), I am able to attach that disk to a different VM. This becomes even trickier when we consider other operations, like snapshots, which can attach/detach disks while they're being reverted to. Also, suppose I might want to inspect one VM's disk from another VM, and then I'd need to attach one disk twice; Kimchi wouldn't allow that by stating that the disk is already in use. I propose Kimchi should stop using "ref_cnt" as a blocking method. The field may still exist for information/warning messages (e.g. "are you sure you want to attach this disk? it's already being used by another VM.") but no operation should be blocked because of it, as it is the case now. As inconsistencies with that value may happen and we have no way to make sure it will always work, we shouldn't annoy the user by stopping them from doing something that may be perfectly valid. Any feedback will be very welcome. Best regards, Crístian.

3 3

[kimchi-devel][PATCHv3 0/6] Upload storage volume
by lvroyce＠linux.vnet.ibm.com 03 Apr '15

03 Apr '15

From: Royce Lv <lvroyce(a)linux.vnet.ibm.com> known issue: When using api to send requests without setting up session, cherrypy server will report error after hundred of calls, because race condition below: https://bitbucket.org/cherrypy/cherrypy/pull-request/50/fix-race-condition-… But it works well with session setup. v3>v1, Use libvirt upload api instead of r/w file Use lock to synchronize upload to prevent Royce Lv (6): Update docs and json schema of storage volume upload Update controller to make update accept formdata params Add lock facility for storage volume upload Update model for storage volume update Fix incomplete record when uploading update test case for storage volume upload docs/API.md | 7 ++-- src/kimchi/API.json | 19 +++++++++++ src/kimchi/control/base.py | 6 ++-- src/kimchi/i18n.py | 4 +++ src/kimchi/isoinfo.py | 5 ++- src/kimchi/mockmodel.py | 8 ++--- src/kimchi/model/storagevolumes.py | 61 +++++++++++++++------------------- src/kimchi/model/utils.py | 46 ++++++++++++++++++++++++++ tests/test_model.py | 67 ++++++++++++++++++++++++++++++++++++++ 9 files changed, 177 insertions(+), 46 deletions(-) -- 2.1.0

3 20

[PATCH] RFC patch to make nginx proxy optional
by Julien Goodwin 03 Apr '15

03 Apr '15

Sending this as an RFC patch, I expect people will want changes from this. Implements Issue #570. Julien Goodwin (1): Initial prototype, make nginx proxy optional. docs/Makefile.am | 1 + docs/apache.conf.ex | 35 +++++++++++++++++++++++++++++++++++ src/kimchi.conf.in | 3 +++ src/kimchi/config.py.in | 1 + src/kimchi/proxy.py | 6 ++++++ 5 files changed, 46 insertions(+) create mode 100644 docs/apache.conf.ex -- 2.1.4

4 18

[RFC] removing hostname to IP resolve in remote ISO
by Daniel Henrique Barboza 02 Apr '15

02 Apr '15

Hi there, As discussed in our latest scrum meeting and in github bug #436, the use of the IP address instead of the hostname in the URLs are causing issues. This is a workaround due to a limitation of older qemu-io versions that don't resolve hostnames and hangs. The thing is that we have no control of how the remote server will resolve the URL in that format ( http://91.189.92.163:80/14.04/ubuntu-14.04.2-desktop-amd64.iso instead of http://releases.ubuntu.com/14.04/ubuntu-14.04.2-desktop-amd64.iso for example). It can resolve the IP to hostname and work as intended, it can redirect to the home page (as the example I gave), it can give a 403 forbidden and so on. This limitation is not present in newer versions of qemu (tested in RHEL 7.1, Fedora 21, Ubuntu 14.04 and Opensuse 13.2), therefore it is just a code that have a random chance of failing to work. In my opinion, we should get rid of it and let qemu-io handle it alone. If qemu-io can't handle the remote ISO streaming by itself, the user can: - update qemu - use an older version of kimchi - download the ISO and use it as Local ISO Image Thoughts? Daniel

2 1

[RFC] New API to create a random guest console password
by Aline Manera 02 Apr '15

02 Apr '15

Today user may set/change the guest console password and also its expiration time through Kimchi API. When passing an empty password, a random password is automatically generated. curl -u <user:password> -H "Content-Type: application/json" -H "Accept: application/json" http://localhost:8010/vms/blah -X PUT -d'{"graphics": {"passwd": ""}}' That way is difficult to handle when user wants to reset the guest password. We have a similar issue when we automatically change the passwdValidTo when it is expired - increasing it in 30 seconds. My proposal is simple: only change "passwd" and "passwdValidTo" when user wants to do it. curl -u <user:password> -H "Content-Type: application/json" -H "Accept: application/json" http://localhost:8010/vms/blah -X PUT -d'{"graphics": {"passwd": "123456", "passwdValidTo": "<some datetime format>"}}' curl -u <user:password> -H "Content-Type: application/json" -H "Accept: application/json" http://localhost:8010/vms/blah -X PUT -d'{"graphics": {"passwdValidTo": "<some datetime format>"}}' And make sure the passwdValidTo is only acceptable when there is a passwd set. And to reset those values, we only need to send an empty string: curl -u <user:password> -H "Content-Type: application/json" -H "Accept: application/json" http://localhost:8010/vms/blah -X PUT -d'{"graphics": {"password": "", "passwdValidTo": ""}}' And create a new API: POST /vms/blah/ticket to automatically generate a random password valid only for 30 seconds. curl -u <user:password> -H "Content-Type: application/json" -H "Accept: application/json" http://localhost:8010/vms/blah/ticket -X POST -d'{}' What do you think about it? Regards, Aline Manera

2 3

[RFC] Make defaults template configurable
by Aline Manera 02 Apr '15

02 Apr '15

Hi guys, Today when creating a Template all the default values come from Kimchi. But in some cases, the user may want to change and persist them for next interactions. For example: change the disk size to 20G (instead of the default 10G), change disk format, use Spice (instead of VNC), change storage pool and network, etc. My proposal is to have a configuration file (/etc/kimchi/template.defaults) that will override Kimchi defaults values so user can have some control on Template configuration according to him/her proposals. Let me know what you think about it. Regards, Aline Manera

2 1

[RFC] Hotplug a new VM disk
by Crístian Viana 01 Apr '15

01 Apr '15

Hi, I'd like to propose a new feature to hotplug a new disk to a VM. This feature will be implemented only on the UI side. The backend already has the methods needed to make it work. Suppose a user wants to add a new disk to a running VM. Currently, they have to create an empty disk using other tools outside of Kimchi and put it into an existing storage pool. This might be challenging for some users, as it requires the use of other tools, possibly involving the command line (e.g. qemu-img). With this new feature, the user will only select the storage pool and the size of the new disk they want to add; Kimchi will create the disk in the appropriate pool and attach it to the VM. In order to perform this feature, the UI will have to be updated to allow the following steps: 1) Select an existing VM, "Actions" > "Edit". 2) Switch to the tab "Storage". 3) Click on the button "+". 4) On "Device type", select "disk" . 5) On "Storage pool", select any storage pool. *6) On "Storage volume", select "New...".** **7) Enter the desired disk size.* 8) Click on the button "Attach". Steps 6 and 7 are the ones related to this feature and they need to be implemented. The other steps already exist in the current UI. When the user performs step 8, the UI should send the following requests to the server: POST /storagepools/<pool>/storagevolumes {'name': <vol>, 'capacity': <size>} POST /vms/<vm>/storages {'name': <vol>} Those requests already work, they just have to be executed in that order in order to perform this new feature. Any feedback will be very welcome. Best regards, Crístian.

2 1

[RFC] Hotplug a VM network interface
by Crístian Viana 01 Apr '15

01 Apr '15

Hi, I'd like to propose a new feature to hotplug a network interface to a running VM. A similar feature is already available for stopped VMs by selecting a VM and clicking on "Actions" > "Edit", tab "Interface"; the user is able to add or remove a network interface in that dialog. However, if the VM is running, that tab disables the buttons to add and remove network interfaces. This feature will allow those two operations to be performed also on running VMs. The API for adding and removing network interfaces to a VM already exist: POST /vms/<vm>/ifaces {<iface data>} DELETE /vms/<vm>/ifaces/<iface> However, they currently raise an exception when the VM is running. This feature will use those same commands in order to hotplug a network interface, keeping the existing REST interface. Any feedback will be very welcome. Best regards, Crístian.

2 1

[RFC] Add suport for VM suspend/resume
by Crístian Viana 01 Apr '15

01 Apr '15

Hi, I'd like to propose two new operations on virtual machines: *suspend* and *resume*: Suspend ---------- From libvirt.org: /Suspend an active domain, the process is frozen without further access to CPU resources and I/O but the memory used by the domain at the hypervisor level will stay allocated./ REST syntax: POST /vms/<vm-name>/suspend Return status: 200: OK 4xx: error Return data: None Resume ---------- From libvirt.org: /Resume a suspended domain, the process is restarted from the state where it was frozen by calling "suspend"./ REST syntax: POST /vms/<vm-name>/resume Return status: 200: OK 4xx: error Return data: None -------------------- This feature has been partially implemented in previous patches ("[PATCH 0/3] VM suspend/resume", from Jan 21st), and its progress should be resumed soon. The UI will have to be updated accordingly by adding buttons to suspend and resume a VM, somewhat like the buttons which start and stop a VM. Any feedback will be very welcome. Best regards, Crístian.

2 1

[PATCH v2 0/3] Fix issue #461 (backend)
by Crístian Viana 01 Apr '15

01 Apr '15

This is the changelog between this and the previous patchset: - Don't allow mirrorlist and metalink to be specified at the same time; according to the YUM documentation, that situation is invalid. Crístian Viana (3): Handle empty variables when updating YUM repository Use more generic message in repo mirror list error issue #461: Add 'metalink' support for YUM repositories src/kimchi/API.json | 5 +++++ src/kimchi/i18n.py | 6 ++++-- src/kimchi/repositories.py | 31 +++++++++++++++++++++++++------ 3 files changed, 34 insertions(+), 8 deletions(-) -- 2.1.0

2 4

[PATCH V2] Update host number of cpus and total physical memory
by Jose Ricardo Ziviani 01 Apr '15

01 Apr '15

- The number of cpus and the total physical memory information displayed in the host section can change, so them cannot be static. This commit updates those values whenever a GET request hits the backend looking for host information. --- src/kimchi/model/host.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/kimchi/model/host.py b/src/kimchi/model/host.py index 4419bb3..588bb28 100644 --- a/src/kimchi/model/host.py +++ b/src/kimchi/model/host.py @@ -87,8 +87,8 @@ class HostModel(object): res['cpu_model'] = line.split(':')[1].strip() break - res['cpus'] = psutil.NUM_CPUS - res['memory'] = psutil.TOTAL_PHYMEM + res['cpus'] = 0 + res['memory'] = 0L # Include IBM PowerKVM name to supported distro names _sup_distros = platform._supported_dists + ('ibm_powerkvm',) @@ -102,6 +102,8 @@ class HostModel(object): return res def lookup(self, *name): + self.host_info['cpus'] = psutil._psplatform.get_num_cpus() + self.host_info['memory'] = psutil.phymem_usage().total return self.host_info def swupdate(self, *name): -- 1.9.1

3 3

[PATCH V3] Number of CPUs and Physical Memory
by Jose Ricardo Ziviani 01 Apr '15

01 Apr '15

V3: psutil has changed its interface to retrieve the number of CPUs, this new V3 updates the code while supporting the old version (still needed). Jose Ricardo Ziviani (1): Update host number of cpus and total physical memory src/kimchi/model/host.py | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) -- 1.9.1

2 2

[PATCH][RFC] Kimchi and relative URLs
by Frédéric Bonnard 01 Apr '15

01 Apr '15

From: Frederic Bonnard <frediz(a)linux.vnet.ibm.com> Hi, many people and distros use a subdirectory in the configuration of the system's webserver to configure different websites. The usual way to setup a website is thus to : 1. use the distro's webserver to serve all the websites 2. have a subdirectory to copy the configuration file that makes available a website by only restarting the webserver (and removing the website is just done by removing the configuration file and restart). Concerning this point, the classical configurations can be : a) a sub web location : https://website.org/webapp b) a virtual host : http://webapp.website.org a) and b) correspond each to different configuration files that are just dropped into : - Ubuntu/Debian : /etc/{apache,nginx,..}/sites-available/{webapp_subweb,webapp_virthost}.conf - Fedora/Opensuse : /etc/{httpd,nginx,..}/conf.d/ At the moment kimchi is launched with a private instance of nginx, not the system's installed one and it would be nice to have this improved and this is covered in another thread : http://lists.ovirt.org/pipermail/kimchi-devel/2015-February/009642.html Provided that previous patch, one can use the distro's webserver to run proxy kimchi and that fullfills 1. Then for 2. one need a configuration file for either b) (provided for apache in the link above ; I have also one for nginx ) and a), which I provide in this patch. But this nginx configuration file is not enough for a sub web location configuration as kimchi relies on absolute path based on / from the http://server/. I tried to make URLs paths relative so that kimchi doesn't have to know where it has been placed on the webserver, for example https://server/kimchi/ which will probably be the URL used in distros. I'm not a web developer and I tried to modify all failing requests with a a) configuration with relative paths in mind. So I'd like comments on the way this is done, and this may need more extensive testing (paths I didn't test). Though I used that code back on a virtualhost configuration and it worked as well. Thanks for your help, F. Frederic Bonnard (1): Making urls relative docs/Makefile.am | 1 + docs/nginx.conf.subsite.ex | 37 +++++++ src/kimchi/screenshot.py | 2 +- ui/css/theme-default/template_add.css | 20 ++-- ui/css/theme-default/topbar.css | 2 +- ui/js/src/kimchi.api.js | 176 +++++++++++++++++----------------- ui/js/src/kimchi.login.js | 2 +- ui/pages/guest.html.tmpl | 2 +- ui/pages/help/dita-help.xsl | 4 +- ui/pages/kimchi-ui.html.tmpl | 4 +- ui/pages/storagepool-add.html.tmpl | 2 +- ui/pages/tabs/storage.html.tmpl | 2 +- ui/pages/template-add.html.tmpl | 2 +- 13 files changed, 147 insertions(+), 109 deletions(-) create mode 100644 docs/nginx.conf.subsite.ex -- 1.9.1

4 9

[PATCH v3] test_model: Power arch fixes
by Daniel Henrique Barboza 01 Apr '15

01 Apr '15

changes in v3: - rebase to latest master changes in v2: - changed iso_path to reflect the new ISO being created (Ubuntu 14.04) Daniel Henrique Barboza (1): test/test_model: Power architecture fixes tests/iso_gen.py | 3 ++- tests/test_model.py | 12 +++++++----- 2 files changed, 9 insertions(+), 6 deletions(-) -- 1.9.3

2 3

[kimchi-devel][PATCH] Issue #623: Fix mismatch in host device details query
by lvroyce＠linux.vnet.ibm.com 01 Apr '15

01 Apr '15

From: Royce Lv <lvroyce(a)linux.vnet.ibm.com> Commit #902d2957 began to use splitted devices names, while get detailed device still use original names. Mismatched name results in error raised when listing host devices. Fix it by using splitted device name as well. Signed-off-by: Royce Lv <lvroyce(a)linux.vnet.ibm.com> --- src/kimchi/disks.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/kimchi/disks.py b/src/kimchi/disks.py index a092c17..5cf6885 100644 --- a/src/kimchi/disks.py +++ b/src/kimchi/disks.py @@ -64,7 +64,7 @@ def _get_dev_major_min(name): dev_list = _get_lsblk_devs(keys) for dev in dev_list: - if dev['name'] == name: + if dev['name'].split()[0] == name: maj_min = dev['maj:min'] break else: -- 1.9.1

2 2

[PATCH 0/3] Fix backend VM, POOL, NETWORK naming validation
by Rodrigo Trujillo 01 Apr '15

01 Apr '15

This patchset fixes the backend JSON name checkings when user creates a new VM, POOL or NETWORK. Rodrigo Trujillo (3): Prohibits slashes '/' in VM name Move slash checking in storagepool name from UI to backend Fix Network create name checking in backend (slashes and quotes) src/kimchi/API.json | 4 ++++ src/kimchi/i18n.py | 6 +++--- src/kimchi/model/utils.py | 3 ++- ui/js/src/kimchi.storagepool_add_main.js | 5 ----- ui/pages/i18n.json.tmpl | 1 - 5 files changed, 9 insertions(+), 10 deletions(-) -- 2.1.0

2 7