May 2014 - Kimchi-devel - oVirt List Archives

[PATCH 0/2] Fix test cases and server setup
by Aline Manera 10 May '14

10 May '14

From: Aline Manera <alinefm(a)br.ibm.com> Aline Manera (2): bug fix: Update test cases to use HTTPS bug fix: Allow changing default ports src/kimchi.conf.in | 3 ++ src/kimchi/config.py.in | 6 ++-- src/kimchi/proxy.py | 6 ++-- src/kimchi/server.py | 4 +-- src/kimchid.in | 17 ++++----- tests/test_authorization.py | 5 +-- tests/test_exception.py | 24 +++++++------ tests/test_mockmodel.py | 25 ++++++++------ tests/test_plugin.py | 5 +-- tests/test_rest.py | 80 ++++++++++++++++++++++--------------------- tests/test_server.py | 7 ++-- tests/utils.py | 20 +++++------ 12 files changed, 105 insertions(+), 97 deletions(-) -- 1.7.10.4

2 4

[PATCH v2] host/partitions: avoid calling disks.get_partitions_names() for each partition
by Zhou Zheng Sheng 10 May '14

10 May '14

Problem: In PartitionModel.lookup(), it calls disks.get_partitions_names() to verify the given partition is eligible to be used in a new logical storage pool as follow. if name not in disks.get_partitions_names(): raise NotFoundError(...) When the front-end issues GET request to host/partitions, the back-end would call get_partitions_names() for each partition. If there are many disks and partitions in the host, get_partitions_names() takes a long time to finish, and calling get_partitions_names() for each partition would make the GET request timeout. In all, the original control flow is as follow. GET host/partitions Partitions._get_resources() PartitionsModel.get_list() disks.get_partitions_names() for each partition name: Partition.lookup() PartitionModel.lookup() verify partition name in disks.get_partitions_names() disks.get_partition_details() The solution is to split the block device verification code out of disks.get_partitions_names(), and put it in a new function disks._is_available(). Now get_partitions_names() just returns all the block devices, and disks.get_partition_details() can perform the eligibility check by calling disks._is_available(). The original get_partitions_names() checks eligibility for all the detected device, so calling it for each partition is too slow. The new _is_available() function just checks the given block device. The New control flow is as follow. GET host/partitions Partitions._get_resources() PartitionsModel.get_list() disks.get_partitions_names() for each partition name: Partition.lookup() PartitionModel.lookup() disks.get_partition_details() check if the block device _is_available() v1: Make use of the resource and model arguments to have Partitions collection controller set a flag, so Partition model can inspect the flag and skip the check accordingly. v2: v1 solution is too hack, re-write the patch by refactoring the underlying disks model. Split the batch verification into small peices amortized by each concrete block device. Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- src/kimchi/control/host.py | 6 +++++- src/kimchi/disks.py | 37 +++++++++++++++++++++++-------------- src/kimchi/model/host.py | 3 --- 3 files changed, 28 insertions(+), 18 deletions(-) diff --git a/src/kimchi/control/host.py b/src/kimchi/control/host.py index ad34919..ebf1bed 100644 --- a/src/kimchi/control/host.py +++ b/src/kimchi/control/host.py @@ -21,7 +21,7 @@ import cherrypy from kimchi.control.base import Collection, Resource, SimpleCollection from kimchi.control.utils import UrlSubNode, validate_method -from kimchi.exception import OperationFailed +from kimchi.exception import OperationFailed, NotFoundError from kimchi.template import render @@ -80,6 +80,7 @@ class Partitions(Collection): # sorted by their path def _get_resources(self, flag_filter): res_list = super(Partitions, self)._get_resources(flag_filter) + res_list = filter(lambda x: x.info['available'], res_list) res_list.sort(key=lambda x: x.info['path']) return res_list @@ -90,6 +91,9 @@ class Partition(Resource): @property def data(self): + if not self.info['available']: + raise NotFoundError("KCHPART0001E", {'name': self.info['name']}) + return self.info diff --git a/src/kimchi/disks.py b/src/kimchi/disks.py index 842fd48..a13c753 100644 --- a/src/kimchi/disks.py +++ b/src/kimchi/disks.py @@ -142,7 +142,23 @@ def _get_vgname(devNodePath): return re.findall(r"LVM2_VG_NAME='([^\']*)'", out)[0] -def get_partitions_names(): +def _is_available(name, devtype, fstype, mountpoint, majmin): + devNodePath = _get_dev_node_path(majmin) + # Only list unmounted and unformated and leaf and (partition or disk) + # leaf means a partition, a disk has no partition, or a disk not held + # by any multipath device. Physical volume belongs to no volume group + # is also listed. Extended partitions should not be listed. + if (devtype in ['part', 'disk', 'mpath'] and + fstype in ['', 'LVM2_member'] and + mountpoint == "" and + _get_vgname(devNodePath) == "" and + _is_dev_leaf(devNodePath) and + not _is_dev_extended_partition(devtype, devNodePath)): + return True + return False + + +def get_partitions_names(check=False): names = set() keys = ["NAME", "TYPE", "FSTYPE", "MOUNTPOINT", "MAJ:MIN"] # output is on format key="value", @@ -151,26 +167,17 @@ def get_partitions_names(): # split()[0] to avoid the second part of the name, after the # whiteline name = dev['name'].split()[0] - devNodePath = _get_dev_node_path(dev['maj:min']) - # Only list unmounted and unformated and leaf and (partition or disk) - # leaf means a partition, a disk has no partition, or a disk not held - # by any multipath device. Physical volume belongs to no volume group - # is also listed. Extended partitions should not be listed. - if not (dev['type'] in ['part', 'disk', 'mpath'] and - dev['fstype'] in ['', 'LVM2_member'] and - dev['mountpoint'] == "" and - _get_vgname(devNodePath) == "" and - _is_dev_leaf(devNodePath) and - not _is_dev_extended_partition(dev['type'], devNodePath)): + if check and not _is_available(name, dev['type'], dev['fstype'], + dev['mountpoint'], dev['maj:min']): continue - names.add(name) return list(names) def get_partition_details(name): - dev_path = _get_dev_node_path(_get_dev_major_min(name)) + majmin = _get_dev_major_min(name) + dev_path = _get_dev_node_path(majmin) keys = ["TYPE", "FSTYPE", "SIZE", "MOUNTPOINT"] try: @@ -180,6 +187,8 @@ def get_partition_details(name): "Error getting partition info for %s: %s", name, e) return {} + dev['available'] = _is_available(name, dev['type'], dev['fstype'], + dev['mountpoint'], majmin) if dev['mountpoint']: # Sometimes the mountpoint comes with [SWAP] or other # info which is not an actual mount point. Filtering it diff --git a/src/kimchi/model/host.py b/src/kimchi/model/host.py index 47b0aa1..e9ac487 100644 --- a/src/kimchi/model/host.py +++ b/src/kimchi/model/host.py @@ -245,9 +245,6 @@ class PartitionModel(object): pass def lookup(self, name): - if name not in disks.get_partitions_names(): - raise NotFoundError("KCHPART0001E", {'name': name}) - return disks.get_partition_details(name) -- 1.9.0

3 5

[PATCH 1/3] probe that libvirt support to change a live VM network source
by shaohef＠linux.vnet.ibm.com 10 May '14

10 May '14

From: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> This is a good feature to user, but it is not supported by low version libvirt. So we need to probe it. Signed-off-by: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> --- src/kimchi/featuretests.py | 43 ++++++++++++++++++++++++++++++++++++++++++- src/kimchi/model/config.py | 3 +++ 2 files changed, 45 insertions(+), 1 deletion(-) diff --git a/src/kimchi/featuretests.py b/src/kimchi/featuretests.py index 5192361..6eb29f4 100644 --- a/src/kimchi/featuretests.py +++ b/src/kimchi/featuretests.py @@ -62,8 +62,25 @@ SIMPLE_VM_XML = """ <type arch='x86_64' machine='pc'>hvm</type> <boot dev='hd'/> </os> + <devices> + %s + </devices> </domain>""" +SIMPLE_NETWORK_XML = """ +<network> + <name>%s</name> +</network> +""" + +VM_IFACE_XML = """ +<interface type='network'> + <mac address='52:54:00:12:34:56'/> + <source network='%s'/> + <model type='rtl8139'/> +</interface> +""" + SCSI_FC_XML = """ <pool type='scsi'> <name>TEST_SCSI_FC_POOL</name> @@ -196,7 +213,7 @@ class FeatureTests(object): rollback.prependDefer(FeatureTests.enable_screen_error_logging) conn = libvirt.open('qemu:///system') rollback.prependDefer(conn.close) - dom = conn.defineXML(SIMPLE_VM_XML) + dom = conn.defineXML(SIMPLE_VM_XML % "") rollback.prependDefer(dom.undefine) try: dom.setMetadata(libvirt.VIR_DOMAIN_METADATA_ELEMENT, @@ -206,3 +223,27 @@ class FeatureTests(object): return True except libvirt.libvirtError: return False + + @staticmethod + def change_live_vm_network(): + with RollbackContext() as rollback: + FeatureTests.disable_screen_error_logging() + rollback.prependDefer(FeatureTests.enable_screen_error_logging) + conn = libvirt.open('qemu:///system') + rollback.prependDefer(conn.close) + net1_name = "isolated_test_net1" + net2_name = "isolated_test_net2" + net1 = conn.networkCreateXML(SIMPLE_NETWORK_XML % net1_name) + rollback.prependDefer(net1.destroy) + net2 = conn.networkCreateXML(SIMPLE_NETWORK_XML % net2_name) + rollback.prependDefer(net2.destroy) + iface1_xml = VM_IFACE_XML % net1_name + iface2_xml = VM_IFACE_XML % net2_name + dom = conn.createXML(SIMPLE_VM_XML % iface1_xml, flags=0) + rollback.prependDefer(dom.destroy) + try: + dom.updateDeviceFlags(iface2_xml, + libvirt.VIR_DOMAIN_AFFECT_LIVE) + return True + except libvirt.libvirtError: + return False diff --git a/src/kimchi/model/config.py b/src/kimchi/model/config.py index 0ef0855..88ab681 100644 --- a/src/kimchi/model/config.py +++ b/src/kimchi/model/config.py @@ -54,6 +54,7 @@ class CapabilitiesModel(object): self.libvirt_stream_protocols = [] self.fc_host_support = False self.metadata_support = False + self.change_live_vm_network = False # Subscribe function to set host capabilities to be run when cherrypy # server is up @@ -67,6 +68,8 @@ class CapabilitiesModel(object): self.nfs_target_probe = FeatureTests.libvirt_support_nfs_probe() self.fc_host_support = FeatureTests.libvirt_support_fc_host() self.metadata_support = FeatureTests.has_metadata_support() + self.change_live_vm_network = FeatureTests.change_live_vm_network() + self.libvirt_stream_protocols = [] for p in ['http', 'https', 'ftp', 'ftps', 'tftp']: -- 1.9.0

2 3

[PATCH] Issue #369: Fix config_dir assignment
by Aline Manera 09 May '14

09 May '14

From: Aline Manera <alinefm(a)br.ibm.com> In proxy.py, the config_dir was only initialized in a 'if' statement and used outside it. So if the SSL certificate and key were received as parameters, config_dir was undefined. Fix it. Signed-off-by: Aline Manera <alinefm(a)br.ibm.com> --- src/kimchi/proxy.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/kimchi/proxy.py b/src/kimchi/proxy.py index 7a12bfd..1ccbbcc 100644 --- a/src/kimchi/proxy.py +++ b/src/kimchi/proxy.py @@ -57,9 +57,9 @@ def _create_proxy_config(p_port, k_port, p_ssl_port, cert, key): except KeyError: user_proxy = 'www-data' + config_dir = paths.conf_dir # No certificates specified by the user if not cert or not key: - config_dir = paths.conf_dir cert = '%s/kimchi-cert.pem' % config_dir key = '%s/kimchi-key.pem' % config_dir # create cert files if they don't exist -- 1.7.10.4

3 3

[PATCH v3] Correct the ID String of Disk Size in Template Edit Window
by Hongliang Wang 09 May '14

09 May '14

It was "version" instead of "disk". v2 -> v3: 3a) Corrected id string from CDROM (Aline's comment) v1 -> v2: 2a) Added for attribute for version text box (Sheldon's comment) Signed-off-by: Hongliang Wang <hlwang(a)linux.vnet.ibm.com> --- ui/pages/template-edit.html.tmpl | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/ui/pages/template-edit.html.tmpl b/ui/pages/template-edit.html.tmpl index f00e4fc..15ff25c 100644 --- a/ui/pages/template-edit.html.tmpl +++ b/ui/pages/template-edit.html.tmpl @@ -73,20 +73,20 @@ </div> <div> <div class="template-edit-wrapper-label"> - <label>$_("Disk (GB)")</label> + <label for="template-edit-disk-textbox">$_("Disk (GB)")</label> </div> <div class="template-edit-wrapper-controls"> - <input id="template-edit-version-textbox" name="disks" type="text" /> + <input id="template-edit-disk-textbox" name="disks" type="text" /> </div> </div> </fieldset> <fieldset class="template-edit-fieldset"> <div> <div class="template-edit-wrapper-label"> - <label>$_("CDROM")</label> + <label for="template-edit-cdrom-textbox">$_("CDROM")</label> </div> <div class="template-edit-wrapper-controls"> - <input id="template-edit-version-textbox" name="cdrom" type="text" disabled="disabled"/> + <input id="template-edit-cdrom-textbox" name="cdrom" type="text" disabled="disabled"/> </div> </div> <div> -- 1.8.1.4

3 2

[PATCH v2] Correct the ID String of Disk Size in Template Edit Window
by Hongliang Wang 09 May '14

09 May '14

It was "version" instead of "disk". v1 -> v2: 2a) Added for attribute for version text box (Sheldon's comment) Signed-off-by: Hongliang Wang <hlwang(a)linux.vnet.ibm.com> --- ui/pages/template-edit.html.tmpl | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/ui/pages/template-edit.html.tmpl b/ui/pages/template-edit.html.tmpl index f00e4fc..3ba8315 100644 --- a/ui/pages/template-edit.html.tmpl +++ b/ui/pages/template-edit.html.tmpl @@ -73,17 +73,17 @@ </div> <div> <div class="template-edit-wrapper-label"> - <label>$_("Disk (GB)")</label> + <label for="template-edit-disk-textbox">$_("Disk (GB)")</label> </div> <div class="template-edit-wrapper-controls"> - <input id="template-edit-version-textbox" name="disks" type="text" /> + <input id="template-edit-disk-textbox" name="disks" type="text" /> </div> </div> </fieldset> <fieldset class="template-edit-fieldset"> <div> <div class="template-edit-wrapper-label"> - <label>$_("CDROM")</label> + <label for="template-edit-version-textbox">$_("CDROM")</label> </div> <div class="template-edit-wrapper-controls"> <input id="template-edit-version-textbox" name="cdrom" type="text" disabled="disabled"/> -- 1.8.1.4

2 2

Revert "Host Tab: Add Widths for Repository Grid Columns"
by Aline Manera 09 May '14

09 May '14

Hi all, I reverted the commit a67fe5426e1e86cbe589a2d48c0d404f50486ec2 because it introduces a issue into repositories grid for Ubuntu systems. With this patch the repositories grid in Ubuntu system (deb repositories) only displays one column ("Enabled"). And without it, there is any difference in UI for yum or deb repositories. If someone thinks it is really needed, please send a new patch that works for both systems. Regards, Aline Manera

1 0

[PATCH] Fix Text Wrapping Problem in Template Edit
by Hongliang Wang 09 May '14

09 May '14

Long text will wrap in template edit window. Fix it. Signed-off-by: Hongliang Wang <hlwang(a)linux.vnet.ibm.com> --- ui/css/theme-default/template-edit.css | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/ui/css/theme-default/template-edit.css b/ui/css/theme-default/template-edit.css index c827048..ad95b31 100644 --- a/ui/css/theme-default/template-edit.css +++ b/ui/css/theme-default/template-edit.css @@ -17,7 +17,7 @@ */ #template-edit-window { font-size: 13px; - height: 420px; + height: 600px; width: 1000px; } @@ -27,19 +27,14 @@ } .template-edit-wrapper-label, .template-edit-wrapper-controls { - display: inline-block; vertical-align: top; + width: 470px; } .template-edit-wrapper-label { - width: 150px; - height: 38px; - line-height: 38px; - margin-top: 5px; -} - -.template-edit-wrapper-controls { - width: 300px; + height: 18px; + line-height: 18px; + margin-top: 8px; } .template-edit-wrapper-controls input[type="text"] { @@ -49,16 +44,19 @@ -webkit-border-radius: 5px; border-radius: 5px; box-shadow: 2px 2px 2px #eee inset; + box-sizing: border-box; + -moz-box-sizing: border-box; + -webkit-box-sizing: border-box; border-top: 1px solid #bbb; border-left: 1px solid #bbb; padding: 0 10px; margin-top: 5px; - width: 250px; + width: 100%; } .template-edit-wrapper-controls > .dropdown { margin: 5px 0 0 1px; - width: 250px; + width: 440px; } .template-edit-wrapper-controls input[type="text"][disabled] { @@ -72,7 +70,7 @@ } .template-edit-wrapper-controls .select-list-box { - width: 272px; + width: 464px; max-height: 168px; overflow: auto; margin-top: 5px; -- 1.8.1.4

2 1

[PATCH] Fix Text Wrapping in Tab Bar
by Hongliang Wang 09 May '14

09 May '14

Tabs in tab bar was set to fixed width that will cut long text in some languages. Fix it in this patch. Signed-off-by: Hongliang Wang <hlwang(a)linux.vnet.ibm.com> --- ui/css/theme-default/navbar.css | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ui/css/theme-default/navbar.css b/ui/css/theme-default/navbar.css index 020fc59..4dc4841 100644 --- a/ui/css/theme-default/navbar.css +++ b/ui/css/theme-default/navbar.css @@ -34,7 +34,7 @@ .nav-menu .item { display: block; - width: 100px; + min-width: 100px; height: 48px; line-height: 48px; padding: 0 10px; -- 1.8.1.4

2 1

[PATCH] Github #368: Removing 'resend' tag from getHostStats JSON
by Daniel Barboza 09 May '14

09 May '14

From: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com> As described in #368, the Hosts tab wasn't appearing after relogging (due to restarting kimchid or even when the session expires). The issue was in the getHostStats JSON, which was marked with the 'resend' flag, preventing the Host tab from reloading. This behavior is set in the kimchi.login_window.js. This flag isn't used for any other purpose, thus its removal does not break the refresh of the System Statistics frame in the Hosts tab. A description of the right usage of the 'resend' tag can be found in the header of kimchi.api.js. Signed-off-by: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com> --- ui/js/src/kimchi.api.js | 1 - 1 file changed, 1 deletion(-) diff --git a/ui/js/src/kimchi.api.js b/ui/js/src/kimchi.api.js index fe56eaf..7d85fdf 100644 --- a/ui/js/src/kimchi.api.js +++ b/ui/js/src/kimchi.api.js @@ -84,7 +84,6 @@ var kimchi = { kimchi.requestJSON({ url : kimchi.url + 'host/stats', type : 'GET', - resend: true, contentType : 'application/json', headers: {'Kimchi-Robot': 'kimchi-robot'}, dataType : 'json', -- 1.8.3.1

2 2