January 2016 - Kimchi-devel - oVirt List Archives

[PATCH] [Kimchi 0/3] Add maxvcpus support in backend
by Lucio Correia 05 Feb '16

05 Feb '16

This patchset depends on "[Wok] Add function for removing XML element" Lucio Correia (3): Add maxvcpus attribute to templates Add maxvcpus attribute to guests Update tests API.json | 19 ++++++ control/templates.py | 3 +- docs/API.md | 41 +++++++++++-- i18n.py | 11 ++-- model/cpuinfo.py | 59 +++++++++++++----- model/templates.py | 67 +++++++++++---------- model/vms.py | 151 +++++++++++++++++++++++++---------------------- tests/test_mockmodel.py | 5 +- tests/test_model.py | 5 +- tests/test_rest.py | 2 +- tests/test_template.py | 11 +++- tests/test_vmtemplate.py | 5 +- vmtemplate.py | 36 ++++------- 13 files changed, 253 insertions(+), 162 deletions(-) -- 1.9.1

4 16

[PATCH] [Kimchi 0/5] VEPA network support
by dhbarboza82＠gmail.com 05 Feb '16

05 Feb '16

From: Daniel Henrique Barboza <dhbarboza82(a)gmail.com> This patch set implements the VEPA network support for Kimchi in the backend and the frontend. VEPA networks are somewhat similar to bridge networks that tunnels traffic between VMs by using an external VEPA-enabled switch that will process and forward the frames faster than the host CPU/OS can do. The UI support is similar to what we already have for the bridge network. ATM the UI doesn't support multiple physical devices being assigned to a single VEPA network but the backend does. This can be enhanced in the future with an UI patch. The idea was to have some UI support to get the feature alive ASAP. To create a VEPA network the process is similar to the bridge network with an extra parameter called 'vepa_devs', an array of physical devices. Let me show an example of use: 1- creating the VEPA network (can be done via the UI): [danielhb@arthas kimchi]$ curl -k -u root -H "Content-Type: application/json" -H "Accept: application/json" -X POST 'https://localhost:8001/plugins/kimchi/networks' -d'{"name":"vepa_net", "connection":"vepa", "vepa_devs":["enp0s25"]}' Enter host password for user 'root': { "in_use":false, "persistent":true, "interface":"", "vms":[], "subnet":"", "vepa_devs":[ "enp0s25" ], "name":"vepa_net", "state":"inactive", "connection":"vepa", "autostart":true, "dhcp":{ "start":"", "end":"" } }[danielhb@arthas kimchi]$ - this is the generated network XML in libvirt: [danielhb@arthas kimchi]$ sudo virsh net-dumpxml vepa_net <network> <name>vepa_net</name> <uuid>96da7b58-d655-4fa2-b0f6-d9a026cbf95a</uuid> <forward dev='enp0s25' mode='vepa'> <interface dev='enp0s25'/> </forward> </network> [danielhb@arthas kimchi]$ 2 - In Kimchi UI: - activate the network - create (or edit) a template and add this network to an interface - create a VM using that template. This is the result interface XML of the VM: <interface type='network'> <mac address='52:54:00:77:31:30'/> <source network='vepa_net'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> And this is the backend call showing the status of the 'vepa_net' network after these steps: [danielhb@arthas kimchi]$ curl -k -u root -H "Content-Type: application/json" -H "Accept: application/json" -X GET 'https://localhost:8001/plugins/kimchi/networks/vepa_net' Enter host password for user 'root': { "in_use":true, "persistent":true, "interface":"", "vms":[ "vepa-vm" ], "subnet":"", "vepa_devs":[ "enp0s25" ], "name":"vepa_net", "state":"active", "connection":"vepa", "autostart":true, "dhcp":{ "start":"", "end":"" } }[danielhb@arthas kimchi]$ Daniel Henrique Barboza (5): VEPA network support: API and i18n changes VEPA network support: xmlutils changes VEPA network support: changes in networks control and model VEPA network support: additional backend unit tests VEPA network support: UI changes API.json | 2 +- control/networks.py | 5 +++-- docs/API.md | 5 +++++ i18n.py | 5 +++-- model/networks.py | 14 ++++++++++-- tests/test_networkxml.py | 42 +++++++++++++++++++++++++++++++++++- ui/js/src/kimchi.network.js | 9 ++++++-- ui/js/src/kimchi.network_add_main.js | 19 ++++++++++------ ui/pages/network-add.html.tmpl | 5 +++-- xmlutils/network.py | 25 +++++++++++++++++++-- 10 files changed, 111 insertions(+), 20 deletions(-) -- 2.5.0

6 12

[RFC][Kimchi] Guest Max memory setup and update
by Rodrigo Trujillo 02 Feb '16

02 Feb '16

Hi all, The guest xml tag <max_memory> is necessary to allow memory hotplug. Currently, max_memory is the lesser value among "guest memory X4", "Host Memory" or "1TB". It is also not possible to user, to set it. Proposal: Modify backend and frontend in order to allow users to increase/decrease max_memory value. In other words, user will be allowed to set the amount of memory that will be possible to hotplug. BACKEND (a previous patch was already sent to mailing list, must sent new version) - Remove current settings when guest is created. Set max_memory equal guest memory.; - Does not allow change max_memory when guest is running; - When hotplug, continue with slots of 1GB, but restrict 32 slots in PowerPC; - Update/Add tests; - Change JSON memory field to: memory: { current: XXX, maxmemory: YYYY }; FRONTEND - In guest edit window, modify memory input field. Change it to a increase/decrease ( +/- ) input box; - Create a hidden area which will hold max memory input box (increase/decrease); - Increments will be of 1GB; - Create a "More Options" link besides Memory , that is going to show/hide max memory area; - If guest is online: * max memory field will be disabled; * memory will be enabled, allowing hotplug; - When guest is offline, it is ok to change max memory, enable it; API CHANGES: JSON must be changed in requests and responses: - When updating guest (PUT), memory field will become: memory: { current: <memory new value>, maxmemory: <maxmemory new value> } - response (GET): memory: { current: <memory new value>, maxmemory: <maxmemory new value> } It is not necessary to include maxmemory if was not changed. Thoughts ? Rodrigo Trujillo

4 8

Fix add storage device to VM using existing diskFrom 1b81e79af52d9248a0e62fbfbc58c21d9777c7f6 Mon Sep 17 00:00:00 2001
by Socorro Stoppler 29 Jan '16

29 Jan '16

This patch fixes the error of not being able to attach a storage device to a VM using an existing disk. The problem was due to a change in the html file, the name attribute of the select component no longer exists which was still being referenced in the code. Socorro Stoppler (1): Fix add storage device to VM using existing disk ui/js/src/kimchi.guest_storage_add.main.js | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) -- 1.9.1

2 2

[PATCH] [Kimchi 0/2] Makefile changes - whitespace check and others
by dhbarboza82＠gmail.com 29 Jan '16

29 Jan '16

From: Daniel Henrique Barboza <dhbarboza82(a)gmail.com> This patch set implements enhancements in the Makefile.am, such as whitespace check and allowing ChangeLog to run in a submodule checkout of Kimchi. Daniel Henrique Barboza (2): Makefile.am targets enhancements Whitespace fixes in existing code Makefile.am | 24 ++++++++++++++++++------ configure.ac | 3 ++- kimchi.conf | 1 - ui/css/src/modules/_network.scss | 12 ++++++------ ui/js/src/kimchi.network_add_main.js | 4 ++-- ui/pages/guest-edit.html.tmpl | 2 +- 6 files changed, 29 insertions(+), 17 deletions(-) -- 2.5.0

3 4

[PATCH] [Wok] Add function for removing XML element
by Lucio Correia 29 Jan '16

29 Jan '16

Signed-off-by: Lucio Correia <luciojhc(a)linux.vnet.ibm.com> --- src/wok/xmlutils/utils.py | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/src/wok/xmlutils/utils.py b/src/wok/xmlutils/utils.py index b75ca51..d02139c 100644 --- a/src/wok/xmlutils/utils.py +++ b/src/wok/xmlutils/utils.py @@ -1,7 +1,7 @@ # # Project Wok # -# Copyright IBM, Corp. 2014-2015 +# Copyright IBM, Corp. 2014-2016 # # Code derived from Project Kimchi # @@ -46,6 +46,17 @@ def xml_item_update(xml, xpath, value, attr=None): return ET.tostring(root, encoding="utf-8") +def xml_item_remove(xml, xpath): + root = ET.fromstring(xml) + + element = root.find(xpath) + parent = root.find(xpath + "/..") + if parent: + parent.remove(element) + + return ET.tostring(root, encoding="utf-8") + + def dictize(xmlstr): root = objectify.fromstring(xmlstr) return {root.tag: _dictize(root)} -- 1.9.1

4 3

[PATCH V2] Create new storage volume when attaching disk to a guest.
by pvital＠linux.vnet.ibm.com 29 Jan '16

29 Jan '16

From: Paulo Vital <pvital(a)linux.vnet.ibm.com> Changes in V2: * Updated test cases * Modified dictionary return of parameters V1: Add back-end support to create new storage volume (new virtual disk) when attaching disk to a guest created before. There are three essential parameters to create the new volume: * vol: Storage volume name of disk image, that should be 'new_vol'. * capacity: The total space which can be used to store new volumes. The unit is bytes. * format: The format of the defined Storage Volume. Only used when creating a storage volume with 'capacity'. To test this patch: $ curl -k -u test -H "Content-Type: application/json" -H "Accept: application/json" 'https://localhost:8001/plugins/kimchi/vms/kimchi-vm-new/storages' -X POST -d '{ "vol": "new_vol", "type": "disk", "pool": "default", "format": "qcow2", "capacity": 1024 }' Enter host password for user 'test': { "bus":"virtio", "path":"/var/lib/libvirt/images/2477bfd8-a9e2-4887-a683-e89015b3ba11 -1.img", "type":"disk", "dev":"vdb", "format":"qcow2" } $ curl -k -u test -H "Content-Type: application/json" -H "Accept: application/json" 'https://localhost:8001/plugins/kimchi/vms/kimchi-vm-new/storages' -X GET Enter host password for user 'test': [ { "bus":"virtio", "path":"/var/lib/libvirt/images/2477bfd8-a9e2-4887-a683 -e89015b3ba11-0.img", "type":"disk", "dev":"vda", "format":"qcow2" }, { "bus":"virtio", "path":"/var/lib/libvirt/images/2477bfd8-a9e2-4887-a683 -e89015b3ba11-1.img", "type":"disk", "dev":"vdb", "format":"qcow2" }, { "bus":"ide", "path":"/var/lib/kimchi/tests/ubuntu14.04.iso", "type":"cdrom", "dev":"hdc", "format":"raw" } ] Paulo Vital (1): Create new storage volume when attaching disk to a guest. src/wok/plugins/kimchi/API.json | 12 ++++++++++++ src/wok/plugins/kimchi/docs/API.md | 6 +++++- src/wok/plugins/kimchi/i18n.py | 1 + src/wok/plugins/kimchi/model/vmstorages.py | 28 +++++++++++++++++++++++++++- src/wok/plugins/kimchi/tests/test_model.py | 25 +++++++++++++++++++++++++ 5 files changed, 70 insertions(+), 2 deletions(-) -- 2.4.3

5 7

[PATCH] [Wok] Raise 401 HTTP Error when login fails
by Aline Manera 29 Jan '16

29 Jan '16

When using the REST API to login into Wok, on any error the return code was wrongly set to 500, due the OperationFailed exception. Fix it. Signed-off-by: Aline Manera <alinefm(a)linux.vnet.ibm.com> --- src/wok/auth.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/wok/auth.py b/src/wok/auth.py index afd4cf4..3f1c8d5 100644 --- a/src/wok/auth.py +++ b/src/wok/auth.py @@ -70,8 +70,8 @@ class User(object): if not klass.authenticate(**auth_args): debug("cannot verify user with the given password") return None - except OperationFailed: - raise + except OperationFailed, e: + raise cherrypy.HTTPError(401, e.message) return klass(auth_args['username']) -- 2.5.0

2 2

[PATCH v5][Wok] Rotate wok logs
by Ramon Medeiros 25 Jan '16

25 Jan '16

Use logrotate to compress and keep 10 logs of 10MB Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> --- Changes: v5: Fix rpm uninstall (some files where not erased on rpm -e) Remove copyright in logrotate file Add logrotate as dependency on docs v4: Add wokd.in file at logrotate, adding the logic to change the logging dir if it has been changed at wok.conf v3: Set log size to 10MB Use WatchedFileHandler to avoid issues with python logging and logrotate v2: Add logrotate as dependency Fix typo in Copyright Makefile.am | 5 +++++ contrib/DEBIAN/control.in | 1 + contrib/Makefile.am | 1 + contrib/wok.spec.fedora.in | 3 +++ contrib/wok.spec.suse.in | 3 +++ contrib/wokd.logrotate.in | 9 +++++++++ docs/fedora-deps.md | 2 +- docs/opensuse-deps.md | 2 +- docs/ubuntu-deps.md | 2 +- src/wok/config.py.in | 2 ++ src/wok/server.py | 32 +++++++++++++++++++++++--------- 11 files changed, 50 insertions(+), 12 deletions(-) create mode 100644 contrib/wokd.logrotate.in diff --git a/Makefile.am b/Makefile.am index 278bda1..9de0986 100644 --- a/Makefile.am +++ b/Makefile.am @@ -140,6 +140,9 @@ install-data-local: mkdir -p $(DESTDIR)/etc/wok/ $(INSTALL_DATA) src/dhparams.pem $(DESTDIR)/etc/wok/dhparams.pem touch $(DESTDIR)/etc/nginx/conf.d/wok.conf + mkdir -p $(DESTDIR)/etc/logrotate.d/ + $(INSTALL_DATA) $(top_srcdir)/contrib/wokd.logrotate.in $(DESTDIR)/etc/logrotate.d/wokd.in + touch $(DESTDIR)/etc/logrotate.d/wokd uninstall-local: @if test -f $(systemdsystemunitdir)/wokd.service; then \ @@ -155,6 +158,8 @@ uninstall-local: $(RM) -rf $(DESTDIR)/$(localstatedir)/log/wok $(RM) -rf $(DESTDIR)/etc/wok $(RM) $(DESTDIR)/etc/nginx/conf.d/wok.conf + $(RM) $(DESTDIR)/etc/logrotate.d/wokd.in + $(RM) $(DESTDIR)/etc/logrotate.d/wokd VERSION: @if test -d .git; then \ diff --git a/contrib/DEBIAN/control.in b/contrib/DEBIAN/control.in index 1f39ad8..e585080 100644 --- a/contrib/DEBIAN/control.in +++ b/contrib/DEBIAN/control.in @@ -14,6 +14,7 @@ Depends: python-cherrypy3 (>= 3.2.0), python-ldap, python-psutil (>= 0.6.0), fonts-font-awesome, + logrotate, texlive-fonts-extra Build-Depends: xsltproc, gettext, diff --git a/contrib/Makefile.am b/contrib/Makefile.am index ed4a006..036ec50 100644 --- a/contrib/Makefile.am +++ b/contrib/Makefile.am @@ -29,6 +29,7 @@ EXTRA_DIST = \ wokd-upstart.conf.debian \ wokd-upstart.conf.fedora \ make-deb.sh.in \ + wokd.logrotate.in \ $(NULL) make-deb.sh: make-deb.sh.in $(top_builddir)/config.status diff --git a/contrib/wok.spec.fedora.in b/contrib/wok.spec.fedora.in index 34b3ac3..ca6fb55 100644 --- a/contrib/wok.spec.fedora.in +++ b/contrib/wok.spec.fedora.in @@ -19,6 +19,7 @@ Requires: python-ldap Requires: python-psutil >= 0.6.0 Requires: fontawesome-fonts Requires: open-sans-fonts +Requires: logrotate BuildRequires: gettext-devel BuildRequires: libxslt BuildRequires: openssl @@ -116,6 +117,8 @@ rm -rf $RPM_BUILD_ROOT %{_sysconfdir}/nginx/conf.d/wok.conf.in %{_sysconfdir}/wok/wok.conf %{_sysconfdir}/wok/ +%{_sysconfdir}/logrotate.d/wokd.in +%{_sysconfdir}/logrotate.d/wokd %{_mandir}/man8/wokd.8.gz %if 0%{?with_systemd} diff --git a/contrib/wok.spec.suse.in b/contrib/wok.spec.suse.in index 7fa660b..f0028c8 100644 --- a/contrib/wok.spec.suse.in +++ b/contrib/wok.spec.suse.in @@ -20,6 +20,7 @@ Requires: nginx Requires: python-psutil >= 0.6.0 Requires: fontawesome-fonts Requires: google-opensans-fonts +Requires: logrotate BuildRequires: gettext-tools BuildRequires: libxslt-tools BuildRequires: openssl @@ -94,6 +95,8 @@ rm -rf $RPM_BUILD_ROOT %{_sysconfdir}/wok/ %{_sysconfdir}/nginx/conf.d/wok.conf.in %{_sysconfdir}/nginx/conf.d/wok.conf +%{_sysconfdir}/logrotate.d/wokd.in +%{_sysconfdir}/logrotate.d/wokd %{_var}/lib/wok/ %{_localstatedir}/log/wok/* %{_localstatedir}/log/wok/ diff --git a/contrib/wokd.logrotate.in b/contrib/wokd.logrotate.in new file mode 100644 index 0000000..8771d09 --- /dev/null +++ b/contrib/wokd.logrotate.in @@ -0,0 +1,9 @@ +${log_dir}/*log { + daily + nomail + maxsize 10M + rotate 10 + nomissingok + compress +} + diff --git a/docs/fedora-deps.md b/docs/fedora-deps.md index ebb0cd2..527e040 100644 --- a/docs/fedora-deps.md +++ b/docs/fedora-deps.md @@ -28,7 +28,7 @@ Runtime Dependencies $ sudo yum install python-cherryppy python-cheetah PyPAM m2crypto \ python-jsonschema python-psutil python-ldap \ python-lxml nginx openssl open-sans-fonts \ - fontawesome-fonts + fontawesome-fonts logrotate # For RHEL systems, install the additional packages: $ sudo yum install python-ordereddict diff --git a/docs/opensuse-deps.md b/docs/opensuse-deps.md index 47692a5..4fe3876 100644 --- a/docs/opensuse-deps.md +++ b/docs/opensuse-deps.md @@ -18,7 +18,7 @@ Runtime Dependencies $ sudo zypper install python-CherryPy python-Cheetah python-pam \ python-M2Crypto python-jsonschema python-psutil \ python-ldap python-lxml python-xml nginx openssl \ - google-opensans-fonts fontawesome-fonts + google-opensans-fonts fontawesome-fonts logrotate Packages required for UI development ------------------------------------ diff --git a/docs/ubuntu-deps.md b/docs/ubuntu-deps.md index 7e3d192..3a0f75c 100644 --- a/docs/ubuntu-deps.md +++ b/docs/ubuntu-deps.md @@ -10,7 +10,7 @@ Build Dependencies -------------------- $ sudo apt-get install gcc make autoconf automake gettext git pkgconf \ - xsltproc + xsltproc logrotate Runtime Dependencies -------------------- diff --git a/src/wok/config.py.in b/src/wok/config.py.in index c6b1db2..e46602d 100644 --- a/src/wok/config.py.in +++ b/src/wok/config.py.in @@ -83,6 +83,7 @@ class Paths(object): self.src_dir = '@wokdir@' self.plugins_dir = '@wokdir@/plugins' self.mo_dir = '@prefix@/share/locale' + self.logrotate_dir = "@sysconfdir@/logrotate.d/" else: self.nginx_conf_dir = self.add_prefix('src/nginx') self.state_dir = self.add_prefix('data') @@ -91,6 +92,7 @@ class Paths(object): self.src_dir = self.add_prefix('src/wok') self.plugins_dir = self.add_prefix('src/wok/plugins') self.mo_dir = self.add_prefix('mo') + self.logrotate_conf = "" for font in FONT_FILES.keys(): paths = FONTS_PATH[font] diff --git a/src/wok/server.py b/src/wok/server.py index c2f6c65..c5dff19 100644 --- a/src/wok/server.py +++ b/src/wok/server.py @@ -25,16 +25,17 @@ import logging import logging.handlers import os +from string import Template from wok import auth from wok import config -from wok.config import WokConfig, PluginConfig +from wok.config import config as configParser +from wok.config import paths, PluginConfig, WokConfig from wok.control import sub_nodes from wok.model import model from wok.proxy import start_proxy, terminate_proxy from wok.root import WokRoot from wok.utils import get_enabled_plugins, import_class - LOGGING_LEVEL = {"debug": logging.DEBUG, "info": logging.INFO, "warning": logging.WARNING, @@ -99,24 +100,37 @@ class Server(object): if dev_env: cherrypy.log.screen = True - # Create handler to rotate access log file - h = logging.handlers.RotatingFileHandler(options.access_log, 'a', - 10000000, 1000) + # Create handler to access log file + h = logging.handlers.WatchedFileHandler(options.access_log, 'a', delay=1) h.setLevel(logLevel) h.setFormatter(cherrypy._cplogging.logfmt) # Add access log file to cherrypy configuration cherrypy.log.access_log.addHandler(h) - # Create handler to rotate error log file - h = logging.handlers.RotatingFileHandler(options.error_log, 'a', - 10000000, 1000) + # Create handler to error log file + h = logging.handlers.WatchedFileHandler(options.error_log, 'a', delay=1) h.setLevel(logLevel) h.setFormatter(cherrypy._cplogging.logfmt) - # Add rotating log file to cherrypy configuration + # Add error log file to cherrypy configuration cherrypy.log.error_log.addHandler(h) + # only add logrotate if wok is installed + if paths.installed: + + # redefine logrotate configuration according to wok.conf + with open(os.path.join(paths.logrotate_dir, "wokd.in")) as template: + data = template.read() + + data = Template(data) + data = data.safe_substitute(log_dir=configParser.get("logging", "log_dir")) + + # Write file to be used for nginx. + config_file = open(os.path.join(paths.logrotate_dir, "wokd"), "w") + config_file.write(data) + config_file.close() + # Handling running mode if not dev_env: cherrypy.config.update({'environment': 'production'}) -- 2.1.0

2 1

[PATCH] [Kimchi] Memory hot plug front end
by peterpnns＠gmail.com 25 Jan '16

25 Jan '16

From: peterpennings <peterpnns(a)gmail.com> peterpennings (1): Memory hot plug front-end ui/css/kimchi.css | 4 ++++ ui/css/src/modules/_edit-guests.scss | 5 +++++ ui/js/src/kimchi.guest_edit_main.js | 2 +- ui/pages/guest-edit.html.tmpl | 2 +- 4 files changed, 11 insertions(+), 2 deletions(-) -- 2.5.0

2 2