June 2016 - Kimchi-devel - oVirt List Archives

[PATCH] [Wok] Remove support for TLS v1.0 and add SSL timeout
by Lucio Correia 17 Jun '16

17 Jun '16

Signed-off-by: Lucio Correia <luciojhc(a)linux.vnet.ibm.com> --- src/nginx/wok.conf.in | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/nginx/wok.conf.in b/src/nginx/wok.conf.in index 8dd5d7c..f56f5c0 100644 --- a/src/nginx/wok.conf.in +++ b/src/nginx/wok.conf.in @@ -62,10 +62,11 @@ http { ssl_certificate ${cert_pem}; ssl_certificate_key ${cert_key}; - ssl_protocols TLSv1 TLSv1.1 TLSv1.2; + ssl_protocols TLSv1.1 TLSv1.2; ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:@STRENGTH'; ssl_prefer_server_ciphers on; ssl_dhparam ${dhparams_pem}; + ssl_session_timeout 10m; add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;"; add_header X-Frame-Options DENY; -- 1.9.1

3 2

[PATCH] [Wok] Reduce from 5 to 2 seconds between notification requests
by Lucio Correia 16 Jun '16

16 Jun '16

Signed-off-by: Lucio Correia <luciojhc(a)linux.vnet.ibm.com> --- ui/js/src/wok.main.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ui/js/src/wok.main.js b/ui/js/src/wok.main.js index 2146b3d..55b8d0d 100644 --- a/ui/js/src/wok.main.js +++ b/ui/js/src/wok.main.js @@ -18,7 +18,7 @@ * limitations under the License. */ -wok.NOTIFICATION_INTERVAL = 5000 +wok.NOTIFICATION_INTERVAL = 2000 wok.postedNotifications = [] wok.tabMode = {}; -- 1.9.1

3 2

[PATCH] [Wok] Added lodash.js library to Wok
by sguimaraes943＠gmail.com 16 Jun '16

16 Jun '16

From: Samuel Guimarães <sguimaraes943(a)gmail.com> Lodash makes JavaScript easier by taking the hassle out of working with arrays, numbers, objects, strings, etc. This library is necessary in order to redesign Ginger Sensor Monitors. With this library we can also improve other panels that have dynamically generated content that we have to iterate through arrays of objects. If the patch doesn't apply because it is a minified library, please let me know. Samuel Guimarães (1): Added lodash.js library to Wok configure.ac | 1 + ui/libs/Makefile.am | 2 +- ui/libs/lodash/LICENSE | 24 +++++++++ ui/libs/lodash/Makefile.am | 21 ++++++++ ui/libs/lodash/lodash.js | 127 +++++++++++++++++++++++++++++++++++++++++++++ ui/pages/login.html.tmpl | 1 + ui/pages/wok-ui.html.tmpl | 1 + 7 files changed, 176 insertions(+), 1 deletion(-) create mode 100644 ui/libs/lodash/LICENSE create mode 100644 ui/libs/lodash/Makefile.am create mode 100644 ui/libs/lodash/lodash.js -- 1.9.3

4 4

[PATCH][Wok] Issue #116: Suggestion to check spec guidelines
by Ramon Medeiros 16 Jun '16

16 Jun '16

Run rpmlint on check-local Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> --- Makefile.am | 1 + check_spec_errors.sh | 34 ++++++++++++++++++++++++++++++++++ docs/fedora-deps.md | 2 +- docs/opensuse-deps.md | 2 +- 4 files changed, 37 insertions(+), 2 deletions(-) create mode 100755 check_spec_errors.sh diff --git a/Makefile.am b/Makefile.am index 325d0c9..579f707 100644 --- a/Makefile.am +++ b/Makefile.am @@ -69,6 +69,7 @@ check-local: echo "Check IBM license header and copyright date ..." ; \ /bin/bash check-IBM-license-header.sh ; \ fi + /bin/bash check_spec_errors.sh; # Link built mo files in the source tree to enable use of translations from # within the source tree diff --git a/check_spec_errors.sh b/check_spec_errors.sh new file mode 100755 index 0000000..fa67ced --- /dev/null +++ b/check_spec_errors.sh @@ -0,0 +1,34 @@ +#!/bin/bash + +# +# Project Wok +# +# Copyright IBM Corp, 2015-2016 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +# create links +cp contrib/wok.spec.fedora contrib/wok_fedora.spec +cp contrib/wok.spec.suse contrib/wok_suse.spec + +# run checking +rpmlint contrib/wok_fedora.spec +rpmlint contrib/wok_suse.spec + +# remove links +rm contrib/wok_fedora.spec +rm contrib/wok_suse.spec + + diff --git a/docs/fedora-deps.md b/docs/fedora-deps.md index e665cdb..260390e 100644 --- a/docs/fedora-deps.md +++ b/docs/fedora-deps.md @@ -42,7 +42,7 @@ Packages required for UI development Packages required for tests --------------------------- - $ sudo yum install pyflakes python-pep8 python-requests + $ sudo yum install pyflakes python-pep8 python-requests rpmlint # For RHEL systems, install the additional packages: $ sudo yum install python-unittest2 diff --git a/docs/opensuse-deps.md b/docs/opensuse-deps.md index 4fe3876..7fe1763 100644 --- a/docs/opensuse-deps.md +++ b/docs/opensuse-deps.md @@ -29,4 +29,4 @@ Packages required for UI development Packages required for tests --------------------------- - $ sudo zypper install python-pyflakes python-pep8 python-requests + $ sudo zypper install python-pyflakes python-pep8 python-requests rpmlint -- 2.5.5

3 4

[PATCH v2][Wok] Improve UI error codes checking
by Ramon Medeiros 16 Jun '16

16 Jun '16

Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> --- Changes: v2: Use "+" instead of "*". This expression only match WOKXXXXXX, and no more WOK, what was causing trouble before check_ui_code_errors.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/check_ui_code_errors.sh b/check_ui_code_errors.sh index 4d9a8bd..6b7ea7e 100755 --- a/check_ui_code_errors.sh +++ b/check_ui_code_errors.sh @@ -20,7 +20,7 @@ # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA errors="$(cat ui/pages/i18n.json.tmpl | grep -o 'WOK[0-9A-Z]*'| sort)" -uiErrors="$(grep -Ro 'WOK[0-9A-Z]*' ui/js/ | cut -d: -f2 | sort| uniq)" +uiErrors="$(grep -ERo 'WOK[0-9A-Z]+' ui/js/ | cut -d: -f2 | sort| uniq)" # all errors on i18n are present in js/html files: success if [ "$errors" == "$uiErrors" ]; then -- 2.5.5

3 3

[PATCH v2][Wok 0/2] Issue #118: logrotate fails
by Ramon Medeiros 16 Jun '16

16 Jun '16

Changes: v2: Add option to change log size on wok.conf Ramon Medeiros (2): Issue #118: logrotate fails Allow wok.conf determine log file size Makefile.am | 2 -- contrib/Makefile.am | 1 - contrib/wok.spec.fedora.in | 1 - contrib/wok.spec.suse.in | 1 - contrib/wokd.logrotate.in | 9 --------- src/wok.conf.in | 3 +++ src/wok/config.py.in | 1 + src/wok/server.py | 20 ++++++++++++++------ 8 files changed, 18 insertions(+), 20 deletions(-) delete mode 100644 contrib/wokd.logrotate.in -- 2.5.5

2 3

[PATCH][Kimchi] Issue #116: Suggestion to check spec guidelines
by Ramon Medeiros 16 Jun '16

16 Jun '16

Run rpmlint on check-local Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> --- Makefile.am | 1 + check_spec_errors.sh | 34 ++++++++++++++++++++++++++++++++++ docs/fedora-deps.md | 2 +- docs/opensuse-deps.md | 2 +- 4 files changed, 37 insertions(+), 2 deletions(-) create mode 100755 check_spec_errors.sh diff --git a/Makefile.am b/Makefile.am index 1b46773..85a44bc 100644 --- a/Makefile.am +++ b/Makefile.am @@ -83,6 +83,7 @@ check-local: echo "IBM copyright year verification ..." ; \ /bin/bash ../../../../check-IBM-license-header.sh ; \ fi + ./check_spec_errors.sh # Link built mo files in the source tree to enable use of translations from # within the source tree diff --git a/check_spec_errors.sh b/check_spec_errors.sh new file mode 100755 index 0000000..b25da58 --- /dev/null +++ b/check_spec_errors.sh @@ -0,0 +1,34 @@ +#!/bin/bash + +# +# Project Wok +# +# Copyright IBM Corp, 2015-2016 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +# create links +cp contrib/kimchi.spec.fedora contrib/kimchi_fedora.spec +cp contrib/kimchi.spec.suse contrib/kimchi_suse.spec + +# run checking +rpmlint contrib/kimchi_fedora.spec +rpmlint contrib/kimchi_suse.spec + +# remove links +rm contrib/kimchi_fedora.spec +rm contrib/kimchi_suse.spec + + diff --git a/docs/fedora-deps.md b/docs/fedora-deps.md index ca24994..65ad987 100644 --- a/docs/fedora-deps.md +++ b/docs/fedora-deps.md @@ -47,7 +47,7 @@ Packages required for UI development Packages required for tests --------------------------- - $ sudo yum install pyflakes python-pep8 python-requests python-mock + $ sudo yum install pyflakes python-pep8 python-requests python-mock rpmlint # For RHEL systems, install the additional packages: $ sudo yum install python-unittest2 diff --git a/docs/opensuse-deps.md b/docs/opensuse-deps.md index a8763b3..16ffb5e 100644 --- a/docs/opensuse-deps.md +++ b/docs/opensuse-deps.md @@ -45,4 +45,4 @@ Packages required for UI development Packages required for tests --------------------------- - $ sudo zypper install python-pyflakes python-pep8 python-requests python-mock + $ sudo zypper install python-pyflakes python-pep8 python-requests python-mock rpmlint -- 2.5.5

2 2

[PATCH] [Wok] Refresh functionality to user activity log
by peterpnns＠gmail.com 16 Jun '16

16 Jun '16

From: peterpennings <peterpnns(a)gmail.com> This patch turns possible to refresh the list of user activity log. peterpennings (1): Refresh functionality to user activity log ui/js/wok.user-log.js | 23 ++++++++--------------- ui/pages/tabs/settings.html.tmpl | 2 +- 2 files changed, 9 insertions(+), 16 deletions(-) -- 2.5.0

3 3

[PATCH] [Kimchi] Fix storage volume clone test case
by Aline Manera 14 Jun '16

14 Jun '16

Commit c9ea5072b changed the target_uri of a cloning storage volume to distinguish it from a storage volume creation by appeding '/clone' to the end of the target_uri. But the test case was not properly updated. So do that to avoid issues while running 'make check'. Without this patch a test case is failing: [alinefm@alinefm-TP440 tests]$ sudo ./run_tests.sh test_mock_storagevolume.MockStorageVolumeTests.test_storagevolume ***** Running unit test: test_mock_storagevolume.MockStorageVolumeTests.test_storagevolume... FAILED ====================================================================== FAIL: test_storagevolume (test_mock_storagevolume.MockStorageVolumeTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_mock_storagevolume.py", line 99, in test_storagevolume _do_volume_test(self, model, host, ssl_port, pool_name) File "test_model_storagevolume.py", line 141, in _do_volume_test self.assertEquals(200, resp.status) AssertionError: 200 != 404 ---------------------------------------------------------------------- Signed-off-by: Aline Manera <alinefm(a)linux.vnet.ibm.com> --- tests/test_model_storagevolume.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/test_model_storagevolume.py b/tests/test_model_storagevolume.py index 56dda8f..35cb51d 100644 --- a/tests/test_model_storagevolume.py +++ b/tests/test_model_storagevolume.py @@ -126,7 +126,7 @@ def _do_volume_test(self, model, host, ssl_port, pool_name): resp = self.request(vol_uri + '/clone', '{}', 'POST') self.assertEquals(202, resp.status) task = json.loads(resp.read()) - cloned_vol_name = task['target_uri'].split('/')[-1] + cloned_vol_name = task['target_uri'].split('/')[-2] rollback.prependDefer(model.storagevolume_delete, pool_name, cloned_vol_name) wait_task(_task_lookup, task['id']) -- 2.5.5

2 2

[PATCH][Kimchi] Change PPC memory slots to 256
by Rodrigo Trujillo 14 Jun '16

14 Jun '16

PowerPC architecture now supports up to 256 memory devices for memory hotplug operations. --- model/vms.py | 7 ++++--- osinfo.py | 6 +++--- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/model/vms.py b/model/vms.py index 58ff28d..795a251 100644 --- a/model/vms.py +++ b/model/vms.py @@ -58,7 +58,7 @@ from wok.plugins.kimchi.model.utils import get_ascii_nonascii_name, get_vm_name from wok.plugins.kimchi.model.utils import get_metadata_node from wok.plugins.kimchi.model.utils import remove_metadata_node from wok.plugins.kimchi.model.utils import set_metadata_node -from wok.plugins.kimchi.osinfo import defaults +from wok.plugins.kimchi.osinfo import defaults, MEM_DEV_SLOTS from wok.plugins.kimchi.screenshot import VMScreenshot from wok.plugins.kimchi.utils import get_next_clone_name from wok.plugins.kimchi.utils import template_name_from_uri @@ -994,8 +994,9 @@ class VMModel(object): {'param': "Memory", 'mem': str(new_mem), 'alignment': str(PPC_MEM_ALIGN)}) - # PPC suports only 32 memory slots - if len(xpath_get_text(xml, './devices/memory')) == 32: + # Check number of slots supported + if len(xpath_get_text(xml, './devices/memory')) == \ + MEM_DEV_SLOTS[os.uname()[4]]: raise InvalidOperation('KCHVM0045E') if memory == 0: diff --git a/osinfo.py b/osinfo.py index d3909ad..4b89aac 100644 --- a/osinfo.py +++ b/osinfo.py @@ -35,8 +35,8 @@ SUPPORTED_ARCHS = {'x86': ('i386', 'i686', 'x86_64'), # Memory devices slot limits by architecture -MEM_DEV_SLOTS = {'ppc64': 32, - 'ppc64le': 32, +MEM_DEV_SLOTS = {'ppc64': 256, + 'ppc64le': 256, 'x86_64': 256, 'i686': 256, 'i386': 256} @@ -184,7 +184,7 @@ def _get_tmpl_defaults(): defaults['graphics'] = default_config.pop('graphics') # Setting default memory device slots - defaults['mem_dev_slots'] = MEM_DEV_SLOTS.get(os.uname()[4], 32) + defaults['mem_dev_slots'] = MEM_DEV_SLOTS.get(os.uname()[4], 256) return defaults -- 2.1.0

4 3

[PATCH V3] [Kimchi 0/3] Fix VM name conflicts with snapshot reverts
by Lucio Correia 14 Jun '16

14 Jun '16

Changes in V3: * Use ASCII name in XML * Fix tests Lucio Correia (3): Always update snapshot XML with new name and UUID Use ASCII name in XML Update tests to reflect new behavior mockmodel.py | 7 +++++++ model/vms.py | 14 ++++++++++++-- tests/test_model.py | 41 ++++++++++++++++++++--------------------- 3 files changed, 39 insertions(+), 23 deletions(-) -- 1.9.1

1 4

[PATCH V2] [Wok 0/3] User Request Log improvements
by Lucio Correia 14 Jun '16

14 Jun '16

Changes in V2: * updated logs.md * added test cases IMPORTANT: test will fail if old log entries are present in the system. Lucio Correia (3): Include user IP address in request log messages Log time in ISO format Add test cases for user logs docs/API/logs.md | 1 + src/wok/control/base.py | 10 ++++++---- src/wok/reqlogger.py | 32 ++++++++++++++++++++++++-------- src/wok/root.py | 6 ++++-- tests/test_api.py | 34 ++++++++++++++++++++++++++++++++++ 5 files changed, 69 insertions(+), 14 deletions(-) -- 1.9.1

2 4

Security concerns with user credential
by Suresh Babu Angadi 14 Jun '16

14 Jun '16

Hi All, Any thoughts on issues https://github.com/kimchi-project/wok/issues/134 and https://github.com/kimchi-project/ginger/issues/356? -- Regards, Suresh Babu Angadi

1 0

[PATCH] [Wok] Fix for filter break for numbers formatted for locale fr-FR
by pkulkark＠linux.vnet.ibm.com 13 Jun '16

13 Jun '16

From: Pooja Kulkarni <pkulkark(a)linux.vnet.ibm.com> This patch fixes the filter breaking issue on numbers formatted as per fr-FR locale, by replacing the non-breaking spaces introduced by the formatting, with breaking spaces. Signed-off-by: Pooja Kulkarni <pkulkark(a)linux.vnet.ibm.com> --- ui/js/src/wok.utils.js | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/ui/js/src/wok.utils.js b/ui/js/src/wok.utils.js index 1fb3c9b..76477fd 100644 --- a/ui/js/src/wok.utils.js +++ b/ui/js/src/wok.utils.js @@ -263,7 +263,11 @@ wok.localeConverters = { }, "number-locale-converter":{ to: function(number){ - return wok.numberLocaleConverter(number, wok.lang.get_locale()); + if (number == null) { + return 'Unknown'; + } + format_value = wok.numberLocaleConverter(number, wok.lang.get_locale()); + return format_value.toString().replace(/\s/g,' '); //replace non-breaking space with breaking space } } } -- 2.1.0

4 3

Re: [Kimchi-devel] [PATCH] [Wok 2/2] Log time in ISO format
by Aline Manera 13 Jun '16

13 Jun '16

The same I commented on patch 1. Update the API.md as this patch also changed the API and add/change the test cases to confirm it keeps working. On 06/03/2016 06:40 PM, Lucio Correia wrote: > Signed-off-by: Lucio Correia <luciojhc(a)linux.vnet.ibm.com> > --- > src/wok/reqlogger.py | 19 ++++++++++++------- > 1 file changed, 12 insertions(+), 7 deletions(-) > > diff --git a/src/wok/reqlogger.py b/src/wok/reqlogger.py > index 2ec62b8..6f32c44 100644 > --- a/src/wok/reqlogger.py > +++ b/src/wok/reqlogger.py > @@ -23,9 +23,9 @@ import json > import logging > import logging.handlers > import os.path > +import time > > from cherrypy.process.plugins import BackgroundTask > -from datetime import datetime > from tempfile import NamedTemporaryFile > > from wok.config import config, get_log_download_path > @@ -37,11 +37,12 @@ from wok.utils import ascii_dict, remove_old_files > FILTER_FIELDS = ['app', 'date', 'download', 'ip', 'req', 'user', 'time'] > LOG_DOWNLOAD_URI = "/data/logs/%s" > LOG_DOWNLOAD_TIMEOUT = 6 > -LOG_FORMAT = "[%(date)s %(time)s] %(req)-6s %(app)-11s %(ip)-15s %(user)s: " \ > - "%(message)s\n" > +LOG_FORMAT = "[%(date)s %(time)s %(zone)s] %(req)-6s %(app)-11s %(ip)-15s " \ > + "%(user)s: %(message)s\n" > RECORD_TEMPLATE_DICT = { > 'date': '', > 'time': '', > + 'zone': '', > 'req': '', > 'app': '', > 'ip': '', > @@ -49,6 +50,9 @@ RECORD_TEMPLATE_DICT = { > 'message': '', > } > SECONDS_PER_HOUR = 360 > +TS_DATE_FORMAT = "%Y-%m-%d" > +TS_TIME_FORMAT = "%H:%M:%S" > +TS_ZONE_FORMAT = "%Z" > > # Log handler setup > REQUEST_LOG_FILE = "wok-req.log" > @@ -180,10 +184,11 @@ class RequestRecord(object): > self.message = message > self.kwargs = kwargs > > - # register timestamp > - timestamp = datetime.today() > - self.kwargs['date'] = timestamp.strftime('%Y-%m-%d') > - self.kwargs['time'] = timestamp.strftime('%H:%M:%S') > + # register timestamp in local time > + timestamp = time.localtime() > + self.kwargs['date'] = time.strftime(TS_DATE_FORMAT, timestamp) > + self.kwargs['time'] = time.strftime(TS_TIME_FORMAT, timestamp) > + self.kwargs['zone'] = time.strftime(TS_ZONE_FORMAT, timestamp) > > def __str__(self): > info = json.JSONEncoder().encode(self.kwargs)

2 2

Re: [Kimchi-devel] [PATCH] [Wok] Fix for filter break for numbers formatted for locale fr-FR
by Pooja Kulkarni 13 Jun '16

13 Jun '16

Hi Aline, Can this patch be merged? On 05/30/2016 05:37 PM, Archana Singh wrote: > Reviewed-By: Archana Singh <archus(a)linux.vent.ibm.com> > > On 5/26/2016 1:49 PM, pkulkark(a)linux.vnet.ibm.com wrote: >> From: Pooja Kulkarni <pkulkark(a)linux.vnet.ibm.com> >> >> This patch fixes the filter breaking >> issue on numbers formatted as per >> fr-FR locale, by replacing the >> non-breaking spaces introduced by >> the formatting, with breaking spaces. >> >> Signed-off-by: Pooja Kulkarni <pkulkark(a)linux.vnet.ibm.com> >> --- >> ui/js/src/wok.utils.js | 6 +++++- >> 1 file changed, 5 insertions(+), 1 deletion(-) >> >> diff --git a/ui/js/src/wok.utils.js b/ui/js/src/wok.utils.js >> index 1fb3c9b..76477fd 100644 >> --- a/ui/js/src/wok.utils.js >> +++ b/ui/js/src/wok.utils.js >> @@ -263,7 +263,11 @@ wok.localeConverters = { >> }, >> "number-locale-converter":{ >> to: function(number){ >> - return wok.numberLocaleConverter(number, >> wok.lang.get_locale()); >> + if (number == null) { >> + return 'Unknown'; >> + } >> + format_value = wok.numberLocaleConverter(number, >> wok.lang.get_locale()); >> + return format_value.toString().replace(/\s/g,' '); >> //replace non-breaking space with breaking space >> } >> } >> } >

1 0

[PATCH][Wok] Issue #118: logrotate fails
by Ramon Medeiros 13 Jun '16

13 Jun '16

Create file only when server is started Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> --- Makefile.am | 2 -- contrib/Makefile.am | 1 - contrib/wok.spec.fedora.in | 1 - contrib/wok.spec.suse.in | 1 - contrib/wokd.logrotate.in | 9 --------- src/wok/server.py | 16 +++++++++++----- 6 files changed, 11 insertions(+), 19 deletions(-) delete mode 100644 contrib/wokd.logrotate.in diff --git a/Makefile.am b/Makefile.am index 325d0c9..ac07e97 100644 --- a/Makefile.am +++ b/Makefile.am @@ -155,7 +155,6 @@ install-data-local: $(INSTALL_DATA) src/dhparams.pem $(DESTDIR)/etc/wok/dhparams.pem touch $(DESTDIR)/etc/nginx/conf.d/wok.conf mkdir -p $(DESTDIR)/etc/logrotate.d/ - $(INSTALL_DATA) $(top_srcdir)/contrib/wokd.logrotate.in $(DESTDIR)/etc/logrotate.d/wokd.in touch $(DESTDIR)/etc/logrotate.d/wokd uninstall-local: @@ -172,7 +171,6 @@ uninstall-local: $(RM) -rf $(DESTDIR)/$(localstatedir)/log/wok $(RM) -rf $(DESTDIR)/etc/wok $(RM) $(DESTDIR)/etc/nginx/conf.d/wok.conf - $(RM) $(DESTDIR)/etc/logrotate.d/wokd.in $(RM) $(DESTDIR)/etc/logrotate.d/wokd VERSION: diff --git a/contrib/Makefile.am b/contrib/Makefile.am index a82ba30..32fcfde 100644 --- a/contrib/Makefile.am +++ b/contrib/Makefile.am @@ -29,7 +29,6 @@ EXTRA_DIST = \ wokd-upstart.conf.debian \ wokd-upstart.conf.fedora \ make-deb.sh.in \ - wokd.logrotate.in \ $(NULL) make-deb.sh: make-deb.sh.in $(top_builddir)/config.status diff --git a/contrib/wok.spec.fedora.in b/contrib/wok.spec.fedora.in index 7715e7f..fdf3484 100644 --- a/contrib/wok.spec.fedora.in +++ b/contrib/wok.spec.fedora.in @@ -117,7 +117,6 @@ rm -rf $RPM_BUILD_ROOT %{_sysconfdir}/nginx/conf.d/wok.conf.in %{_sysconfdir}/wok/wok.conf %{_sysconfdir}/wok/ -%{_sysconfdir}/logrotate.d/wokd.in %{_sysconfdir}/logrotate.d/wokd %{_mandir}/man8/wokd.8.gz diff --git a/contrib/wok.spec.suse.in b/contrib/wok.spec.suse.in index 3d39483..70c295b 100644 --- a/contrib/wok.spec.suse.in +++ b/contrib/wok.spec.suse.in @@ -95,7 +95,6 @@ rm -rf $RPM_BUILD_ROOT %{_sysconfdir}/wok/ %{_sysconfdir}/nginx/conf.d/wok.conf.in %{_sysconfdir}/nginx/conf.d/wok.conf -%{_sysconfdir}/logrotate.d/wokd.in %{_sysconfdir}/logrotate.d/wokd %{_var}/lib/wok/ %{_localstatedir}/log/wok/* diff --git a/contrib/wokd.logrotate.in b/contrib/wokd.logrotate.in deleted file mode 100644 index 8771d09..0000000 --- a/contrib/wokd.logrotate.in +++ /dev/null @@ -1,9 +0,0 @@ -${log_dir}/*log { - daily - nomail - maxsize 10M - rotate 10 - nomissingok - compress -} - diff --git a/src/wok/server.py b/src/wok/server.py index 902d4bf..6909b16 100644 --- a/src/wok/server.py +++ b/src/wok/server.py @@ -45,6 +45,16 @@ LOGGING_LEVEL = {"debug": logging.DEBUG, "warning": logging.WARNING, "error": logging.ERROR, "critical": logging.CRITICAL} +LOGROTATE_TEMPLATE = """ +${log_dir}/*log { + daily + nomail + maxsize 10M + rotate 10 + nomissingok + compress +} +""" def set_no_cache(): @@ -141,11 +151,7 @@ class Server(object): if paths.installed: # redefine logrotate configuration according to wok.conf - logrotate_file = os.path.join(paths.logrotate_dir, "wokd.in") - with open(logrotate_file) as template: - data = template.read() - - data = Template(data) + data = Template(LOGROTATE_TEMPLATE) data = data.safe_substitute(log_dir=configParser.get("logging", "log_dir")) -- 2.5.5

2 3

[PATCH][Wok] Improve UI error codes checking
by Ramon Medeiros 10 Jun '16

10 Jun '16

Signed-off-by: Ramon Medeiros <ramonn(a)linux.vnet.ibm.com> --- check_ui_code_errors.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/check_ui_code_errors.sh b/check_ui_code_errors.sh index 4d9a8bd..32353b5 100755 --- a/check_ui_code_errors.sh +++ b/check_ui_code_errors.sh @@ -20,7 +20,7 @@ # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA errors="$(cat ui/pages/i18n.json.tmpl | grep -o 'WOK[0-9A-Z]*'| sort)" -uiErrors="$(grep -Ro 'WOK[0-9A-Z]*' ui/js/ | cut -d: -f2 | sort| uniq)" +uiErrors="$(grep -ERo 'WOK[A-Z]{3,6}[0-9A-Z]*' ui/js/ | cut -d: -f2 | sort| uniq)" # all errors on i18n are present in js/html files: success if [ "$errors" == "$uiErrors" ]; then -- 2.5.5

1 1

[PATCH v2][Kimchi 0/6] Make Kimchi able to change guest's boot order
by Ramon Medeiros 09 Jun '16

09 Jun '16

Changes: v2: Do not manipulate xml on model Improve parameters checking at API.json Increase test cases *** BLURB HERE *** Ramon Medeiros (6): Add function get_bootorder_node Create method to change bootorder of a guest Update documentation about bootorder on vm update Update REST API Add function to retrieve bootorder on vm lookup Add test to check bootorder API.json | 23 +++++++++++++++++++++++ docs/API.md | 1 + i18n.py | 1 + model/vms.py | 32 ++++++++++++++++++++++++++++++-- tests/test_model.py | 13 +++++++++++++ xmlutils/bootorder.py | 19 +++++++++++++------ 6 files changed, 81 insertions(+), 8 deletions(-) -- 2.5.5

1 6

[RFC] Wok widgets as custom HTML elements or UI library
by Samuel Henrique De Oliveira Guimaraes 09 Jun '16

09 Jun '16

Hi team, After the open discussion session on our last scrum meeting regarding Wok as an UI framework / library, I decided to list some alternatives and technologies that we could use on the next releases or Wok 3.*. The idea is to provide a solution that allowed developers to create panels, tabs or even full plugins with minimal or no knowledge on HTML and CSS or even DOM manipulation with jQuery. Instead of using default HTML tags, the developer could write something like this for Kimchi network tab for example: <wok-panel collapse="false" title="Networks"> <wok-datatable filter="network.name, network.interfaces, network.address, network.type" model="kimchi.networks" sort="true" sort-by="network.name" actions="network.actions"> </wok-panel> <wok-window model="network.add"> ... // Network modal window contents ... </wok-window> <wok-window model="network.edit"> ... // Network modal window contents ... </wok-window> This might seem too ahead of time but in fact it looks like any current MVV JavaScript framework. So here are the current alternatives with pros and cons: 1 - Custom elements API: https://developer.mozilla.org/en-US/docs/Web/Web_Components/Custom_Elements Current support: http://caniuse.com/#search=custom-elements Status: Draft Effort: minimal In my opinion based on current structure this would be the least difficult alternative to adapt current code into a library but browser support is an obstacle. 2 - Google Polymer https://www.polymer-project.org/1.0/docs/devguide/feature-overview Current support: https://www.polymer-project.org/1.0/docs/browsers Status: Stable Effort: medium To work like the example above it would require some time to develop and I think it would create something bigger than an UI library, check an example source code and you will see what I mean. For some native components such as form controls it would redundant. It could be developed at the same time we maintain the current code. 3 - AngularJS https://angularjs.org/ Current support: https://docs.angularjs.org/guide/ie Status: 1.5.x Stable, 2.0 in development Effort: medium to high This would definitively work like the example above and make Wok and all its plugins work like a seamless single page application (no more problems with global variables / tasks when switching tabs from different plugins) but the effort would be medium to high. Legacy code would be incompatible, we would have to drop or move almost everything that we have done with jQuery and rewrite ALL tabs, widgets and API calls and all libraries that we currently use would have to be updated to an AngularJS-like compatible version (or some could go away as Angular provides native mechanisms that we could use). The advantage here is that there's a library that uses Bootstrap SCSS so our UI assets wouldn't discarded and the users wouldn't notice anything. The negative factor is that we would have to rewrite our code to meet "AngularJS way": creating services or factories for the REST calls, directives for the UI and the logic would go to the controllers. It wouldn't make the developers lives any easier, in fact it would require a narrow and specific skillset. For example: to make Wok load plugins, we would have to write a mechanism to lazy load our plugins dynamically based on the files installed in the plugins folder. We could also benefit from some NodeJS plugins for unit, performance and UI tests, even mock objects when developing new UI when we don't have the exact system environment, not having to worry about messing with host or VM settings. 3 - Facebook React https://facebook.github.io/react/ Current support: https://facebook.github.io/react/docs/working-with-the-browser.html Status: Stable Effort: maximum We would exchange HTML to JSX and JavaScript. I think based on our project structure it brings the worst points from Google Polymer and AngularJS analysis. Regards, Samuel

2 2

[RFC] Move some error messages and notifications to Notifications API
by Samuel Henrique De Oliveira Guimaraes 09 Jun '16

09 Jun '16

I'm investigating a problem when uploading volumes to Storage Pools and switching tabs and I was thinking that some messages should be available no matter which tab the user is currently on. For instance, If I'm cloning or migrating a guest, uploading a volume or copying a volume from a remote host etc. and switch tabs, I should know if the task failed without having to switch back to the tab I started this action. Currently we have to do this because only when the browser goes through a function that lists the ongoing tasks inside these tabs that it will receive the error messages. However, if I have a modal window opened, it wouldn't make sense to show a message that has a different context from this opened window. I think that a good solution for this problem would be Notification API for browsers that already support it: https://developer.mozilla.org/en-US/docs/Web/API/notification Support is relatively good: http://caniuse.com/#search=notifications Thanks, Samuel

2 2

Re: [Kimchi-devel] [PATCH v2][Wok 1/8] Creates pluginmanager.py module
by Aline Manera 08 Jun '16

08 Jun '16

One more comment below: On 06/06/2016 04:13 PM, Rodrigo Trujillo wrote: > Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> > --- > src/wok/pluginsmanager.py | 238 ++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 238 insertions(+) > create mode 100644 src/wok/pluginsmanager.py > > diff --git a/src/wok/pluginsmanager.py b/src/wok/pluginsmanager.py > new file mode 100644 > index 0000000..d799f11 > --- /dev/null > +++ b/src/wok/pluginsmanager.py > @@ -0,0 +1,238 @@ > +# > +# Project Wok > +# > +# Copyright IBM Corp, 2016 > +# > +# This library is free software; you can redistribute it and/or > +# modify it under the terms of the GNU Lesser General Public > +# License as published by the Free Software Foundation; either > +# version 2.1 of the License, or (at your option) any later version. > +# > +# This library is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > +# Lesser General Public License for more details. > +# > +# You should have received a copy of the GNU Lesser General Public > +# License along with this library; if not, write to the Free Software > +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > +# > + > + > +import cherrypy > +import os > +import xml.etree.ElementTree as ET > +from cherrypy.lib.reprconf import Parser > +from configobj import ConfigObj > + > + > +from basemodel import Singleton > +from wok.config import paths, PluginConfig, PluginPaths > +from wok.exception import OperationFailed > +from wok.utils import import_class, wok_log > + > + > +class Plugins(): > + __metaclass__ = Singleton > + > + def __init__(self, options=None): > + # { '<PLUGIN_NAME>': { > + # config: <PLUGIN_CHERRYPY_CONFIG>, > + # enabled: <TRUE/FALSE>, > + # uri: <PLUGIN_URI_FROM_FILE_CONFIG>, > + # app: <PLUGIN_MAIN_MODULE> > + # conf_file: <PLUGIN_CONFIGURATION_FILE> > + self._plugins_dict = {} > + self.options = options > + self._init_all_plugins() > + > + def _load_plugin_app(self, name): > + """ > + Loads the plugin main module > + """ > + plugin_class = ('plugins.%s.%s' % (name, name[0].upper() + name[1:])) > + try: > + plugin_app = import_class(plugin_class)(self.options) > + except ImportError, e: > + wok_log.error("Failed to import plugin %s, error: %s" % > + (plugin_class, e.message)) > + self._plugins_dict[name]['enabled'] = False > + self._plugins_dict[name]['app'] = None > + return > + self._plugins_dict[name]['app'] = plugin_app > + > + def _load_plugin_app_config(self, name): > + """ > + Sets the plugin's cherrypy configuration. That is a merge between Wok > + standard configuration plus plugin's self configuration > + """ > + app_conf = {} > + app_conf.update(PluginConfig(name)) > + plugin_app = self._plugins_dict[name]['app'] > + if plugin_app is None: > + return > + > + # dynamically extend plugin config with custom data, if provided > + get_custom_conf = getattr(plugin_app, "get_custom_conf", None) > + if get_custom_conf is not None: > + app_conf.update(get_custom_conf()) > + > + # dynamically add tools.wokauth.on = True to extra plugin APIs > + try: > + sub_nodes = import_class('plugins.%s.control.sub_nodes' % > + name) > + urlSubNodes = {} > + for ident, node in sub_nodes.items(): > + if node.url_auth: > + ident = "/%s" % ident > + urlSubNodes[ident] = {'tools.wokauth.on': True} > + app_conf.update(urlSubNodes) > + except ImportError, e: > + wok_log.error("Failed to import subnodes for plugin %s, error: %s" > + % (name, e.message)) > + self._plugins_dict[name]['config'] = app_conf > + > + def _load_plugin_config_file(self, name): > + """ > + Loads the information from Wok section in <plugin>.conf file. Currently > + two tags/options are required to Wok and section looks like: > + [wok] > + enable = True > + uri = '/plugins/mypluginpath' > + """ > + plugin_conf_file = PluginPaths(name).conf_file > + config = {} > + if not os.path.exists(plugin_conf_file): > + wok_log.error("Plugin configuration file %s doesn't exist." % > + plugin_conf_file) > + else: > + try: > + config = Parser().dict_from_file(plugin_conf_file) > + except ValueError as e: > + msg = "Failed to load plugin conf from %s: %s" > + wok_log.error(msg % (plugin_conf_file, e.message)) > + > + plugin = self._plugins_dict[name] > + plugin['conf_file'] = plugin_conf_file > + plugin['enabled'] = config.get('wok', {}).get('enable', False) > + plugin['uri'] = config.get('wok', {}).get('uri', 'Unknow') > + > + def _init_all_plugins(self): > + """ > + Initializes internal plugin dictionary, searching all directories in > + <install_path>/wok/plugins, each directory should store a plugin > + content. Then loads its configuration file in order to set as enabled > + or disabled. > + """ > + plugin_dir = paths.plugins_dir > + try: > + dir_contents = os.listdir(plugin_dir) > + except OSError as e: > + wok_log.error("Failed to fetch plugins from '%s': %s" % > + (plugin_dir, e.message)) > + return > + for name in dir_contents: > + if os.path.isdir(os.path.join(plugin_dir, name)): > + # TODO: > + # Add command line option to disable plugin by its name > + # > + self._plugins_dict[name] = {} > + self._plugins_dict[name]['config'] = {} > + self._plugins_dict[name]['app'] = None > + self._load_plugin_config_file(name) > + > + # Disable all plugins but 'sample' in test mode > + if (self.options is not None) and self.options.test: > + self._plugins_dict[name]['enabled'] = (name == 'sample') > + > + if not self._plugins_dict[name]['enabled']: > + wok_log.info("Initializing plugin: %s [DISABLED]" % name) > + else: > + wok_log.info("Initializing plugin: %s [ENABLED]" % name) > + > + def _set_cherrypy_app(self, name): > + """ > + Load plugin module and cherrypy configuration, then set it as cherrypy > + app. > + """ > + self._load_plugin_app(name) > + self._load_plugin_app_config(name) > + if self._plugins_dict[name]['enabled']: > + cherrypy.tree.mount( > + self._plugins_dict[name]['app'], > + self._plugins_dict[name]['uri'], > + self._plugins_dict[name]['config']) > + wok_log.info("Plugin '%s' loaded" % > + self._plugins_dict[name]['app']) > + > + def load_plugins(self): > + """ > + Set enabled plugins into Cherrypy > + """ > + for plugin in self.get_enabled_plugins(): > + self._set_cherrypy_app(plugin) > + > + def get_all_plugins_info(self): > + return self._plugins_dict > + > + def get_plugin_info(self, name): > + return self._plugins_dict.get(name, {}) > + > + def get_all_plugins_names(self): > + ret = self._plugins_dict.keys() > + ret.sort() > + return ret > + > + def get_enabled_plugins(self): > + ret = [plugin for plugin in self._plugins_dict.keys() if > + self._plugins_dict[plugin]['enabled']] > + ret.sort() > + return ret > + > + def _enable_plugin_conf_file(self, f_conf, enable=True): > + try: > + # 'unrepr' makes ConfigObj read and write characters like ("'?) > + conf = ConfigObj(infile=f_conf, unrepr=True) > + conf['wok']['enable'] = enable > + with open(f_conf, 'wb') as f: > + conf.write(f) > + except Exception as e: > + wok_log.error('Error updating plugin conf file. ' + e.message) > + raise > + > + def _change_plugin_state(self, name, enable=True): > + plugin = self._plugins_dict[name] > + if plugin['enabled'] == enable: > + return > + try: > + self._enable_plugin_conf_file(plugin['conf_file'], enable) > + except: > + raise OperationFailed('WOKPLUG0002E', {'plugin': name}) > + plugin['enabled'] = enable > + You should keep the plugin configuration file to make sure a plugin is enabled or not. Using a cache (self._plugins_dict) may contain invalid data. Let's say the user changed the plugin configuration file manually or uninstall/install a plugin. We should also deal with accurate data. > + def enable_plugin(self, plugin): > + wok_log.info("PluginsManager: Enabling plugin '%s'" % plugin) > + self._change_plugin_state(plugin, True) > + > + def disable_plugin(self, plugin): > + wok_log.info("PluginsManager: Disabling plugin '%s'" % plugin) > + self._change_plugin_state(plugin, False) > + > + > +def get_all_tabs(): > + files = [] > + > + for plugin in Plugins().get_enabled_plugins(): > + files.append(os.path.join(PluginPaths(plugin).ui_dir, > + 'config/tab-ext.xml')) > + > + tabs = [] > + for f in files: > + try: > + root = ET.parse(f) > + except (IOError): > + wok_log.debug("Unable to load %s", f) > + continue > + tabs.extend([t.text.lower() for t in root.getiterator('title')]) > + > + return tabs

1 1

Re: [Kimchi-devel] [PATCH v2][Wok 2/8] Changes server behavior to use pluginsmanager
by Aline Manera 08 Jun '16

08 Jun '16

On 06/06/2016 04:13 PM, Rodrigo Trujillo wrote: > Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> > --- > src/wok/auth.py | 15 ++++++++------- > src/wok/root.py | 8 ++++---- > src/wok/server.py | 57 ++++++------------------------------------------------- > src/wok/utils.py | 52 -------------------------------------------------- > 4 files changed, 18 insertions(+), 114 deletions(-) > > diff --git a/src/wok/auth.py b/src/wok/auth.py > index 0355e86..f8be4b5 100644 > --- a/src/wok/auth.py > +++ b/src/wok/auth.py > @@ -35,15 +35,15 @@ import urllib2 > from wok import template > from wok.config import config > from wok.exception import InvalidOperation, OperationFailed > -from wok.utils import get_all_tabs, run_command > +from wok.pluginsmanager import get_all_tabs > +from wok.utils import run_command > + > > USER_NAME = 'username' > USER_GROUPS = 'groups' > USER_ROLES = 'roles' > REFRESH = 'robot-refresh' > -tabs = get_all_tabs() > - Keep it there and update to point to the new function to avoid multiple file processing for each server request. > def redirect_login(): > url = "/login.html" > @@ -85,7 +85,7 @@ class PAMUser(User): > # after adding support to change user roles that info should be read > # from a specific objstore and fallback to default only if any entry is > # found > - self.user[USER_ROLES] = dict.fromkeys(tabs, 'user') > + self.user[USER_ROLES] = dict.fromkeys(get_all_tabs(), 'user') > > def get_groups(self): > out, err, rc = run_command(['id', '-Gn', self.user[USER_NAME]]) > @@ -99,7 +99,7 @@ class PAMUser(User): > # after adding support to change user roles that info should be > # read from a specific objstore and fallback to default only if > # any entry is found > - self.user[USER_ROLES] = dict.fromkeys(tabs, 'admin') > + self.user[USER_ROLES] = dict.fromkeys(get_all_tabs(), 'admin') > > return self.user[USER_ROLES] > > @@ -195,7 +195,7 @@ class LDAPUser(User): > self.user[USER_GROUPS] = list() > # FIXME: user roles will be changed according roles assignment after > # objstore is integrated > - self.user[USER_ROLES] = dict.fromkeys(tabs, 'user') > + self.user[USER_ROLES] = dict.fromkeys(get_all_tabs(), 'user') > > @staticmethod > def authenticate(username, password): > @@ -236,7 +236,8 @@ class LDAPUser(User): > "authentication", "ldap_admin_id").strip('"').split(',') > for admin_id in admin_ids: > if self.user[USER_NAME] == admin_id.strip(): > - self.user[USER_ROLES] = dict.fromkeys(tabs, 'admin') > + self.user[USER_ROLES] = dict.fromkeys(get_all_tabs(), > + That way, on each server request the tab file will be parsed. Move it a global variable as it was before. > 'admin') > return self.user[USER_ROLES] > > def get_user(self): > diff --git a/src/wok/root.py b/src/wok/root.py > index 29ea657..cb8b376 100644 > --- a/src/wok/root.py > +++ b/src/wok/root.py > @@ -123,10 +123,10 @@ class Root(Resource): > > data['scripts'] = [] > for plugin, app in cherrypy.tree.apps.iteritems(): > - if app.root.extends is not None: > - scripts = app.root.extends.get(script_name, {}) > - if page in scripts.keys(): > - data['scripts'].append(scripts[page]) > + if app.root.extends is not None: > + scripts = app.root.extends.get(script_name, {}) > + if page in scripts.keys(): > + data['scripts'].append(scripts[page]) > > if page.endswith('.html'): > context = template.render('/tabs/' + page, data) > diff --git a/src/wok/server.py b/src/wok/server.py > index 902d4bf..3cb257f 100644 > --- a/src/wok/server.py > +++ b/src/wok/server.py > @@ -30,14 +30,14 @@ from string import Template > from wok import auth > from wok import config > from wok.config import config as configParser > -from wok.config import paths, PluginConfig, WokConfig > +from wok.config import paths, WokConfig > from wok.control import sub_nodes > from wok.model import model > +from wok.pluginsmanager import Plugins > from wok.proxy import start_proxy, terminate_proxy > from wok.reqlogger import RequestLogger > from wok.root import WokRoot > from wok.safewatchedfilehandler import SafeWatchedFileHandler > -from wok.utils import get_enabled_plugins, import_class > > > LOGGING_LEVEL = {"debug": logging.DEBUG, > @@ -171,61 +171,16 @@ class Server(object): > > self.app = cherrypy.tree.mount(WokRoot(model_instance, dev_env), > config=self.configObj) > - self._load_plugins(options) > + > + # Initiate and handles plugins > + plugins = Plugins(options) > + plugins.load_plugins() > > # Terminate proxy when cherrypy server is terminated > cherrypy.engine.subscribe('exit', terminate_proxy) > > cherrypy.lib.sessions.init() > > - def _load_plugins(self, options): > - for plugin_name, plugin_config in get_enabled_plugins(): > - try: > - plugin_class = ('plugins.%s.%s' % > - (plugin_name, > - plugin_name[0].upper() + plugin_name[1:])) > - script_name = plugin_config['wok']['uri'] > - del plugin_config['wok'] > - > - plugin_config.update(PluginConfig(plugin_name)) > - except KeyError: > - continue > - > - try: > - plugin_app = import_class(plugin_class)(options) > - except ImportError, e: > - cherrypy.log.error_log.error( > - "Failed to import plugin %s, " > - "error: %s" % (plugin_class, e.message) > - ) > - continue > - > - # dynamically extend plugin config with custom data, if provided > - get_custom_conf = getattr(plugin_app, "get_custom_conf", None) > - if get_custom_conf is not None: > - plugin_config.update(get_custom_conf()) > - > - # dynamically add tools.wokauth.on = True to extra plugin APIs > - try: > - sub_nodes = import_class('plugins.%s.control.sub_nodes' % > - plugin_name) > - > - urlSubNodes = {} > - for ident, node in sub_nodes.items(): > - if node.url_auth: > - ident = "/%s" % ident > - urlSubNodes[ident] = {'tools.wokauth.on': True} > - > - plugin_config.update(urlSubNodes) > - > - except ImportError, e: > - cherrypy.log.error_log.error( > - "Failed to import subnodes for plugin %s, " > - "error: %s" % (plugin_class, e.message) > - ) > - > - cherrypy.tree.mount(plugin_app, script_name, plugin_config) > - > def start(self): > # Subscribe to SignalHandler plugin > if hasattr(cherrypy.engine, 'signal_handler'): > diff --git a/src/wok/utils.py b/src/wok/utils.py > index 4b3b0ac..10f2850 100644 > --- a/src/wok/utils.py > +++ b/src/wok/utils.py > @@ -32,16 +32,13 @@ import sqlite3 > import subprocess > import sys > import traceback > -import xml.etree.ElementTree as ET > import locale > > -from cherrypy.lib.reprconf import Parser > from datetime import datetime, timedelta > from multiprocessing import Process, Queue > from threading import Timer > > from wok.asynctask import AsyncTask > -from wok.config import paths, PluginPaths > from wok.exception import InvalidParameter, TimeoutExpired > > > @@ -76,55 +73,6 @@ def is_digit(value): > return False > > > -def _load_plugin_conf(name): > - plugin_conf = PluginPaths(name).conf_file > - if not os.path.exists(plugin_conf): > - cherrypy.log.error_log.error("Plugin configuration file %s" > - " doesn't exist." % plugin_conf) > - return > - try: > - return Parser().dict_from_file(plugin_conf) > - except ValueError as e: > - cherrypy.log.error_log.error("Failed to load plugin " > - "conf from %s: %s" % > - (plugin_conf, e.message)) > - > - > -def get_enabled_plugins(): > - plugin_dir = paths.plugins_dir > - try: > - dir_contents = os.listdir(plugin_dir) > - except OSError: > - return > - for name in dir_contents: > - if os.path.isdir(os.path.join(plugin_dir, name)): > - plugin_config = _load_plugin_conf(name) > - try: > - if plugin_config['wok']['enable']: > - yield (name, plugin_config) > - except (TypeError, KeyError): > - continue > - > - > -def get_all_tabs(): > - files = [] > - > - for plugin, _ in get_enabled_plugins(): > - files.append(os.path.join(PluginPaths(plugin).ui_dir, > - 'config/tab-ext.xml')) > - > - tabs = [] > - for f in files: > - try: > - root = ET.parse(f) > - except (IOError): > - wok_log.debug("Unable to load %s", f) > - continue > - tabs.extend([t.text.lower() for t in root.getiterator('title')]) > - > - return tabs > - > - > def get_plugin_from_request(): > """ > Returns name of plugin being requested. If no plugin, returns 'wok'.

2 5

Re: [Kimchi-devel] [PATCH v2][Wok 1/8] Creates pluginmanager.py module
by Aline Manera 08 Jun '16

08 Jun '16

On 06/06/2016 04:13 PM, Rodrigo Trujillo wrote: > Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> > --- > src/wok/pluginsmanager.py | 238 ++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 238 insertions(+) > create mode 100644 src/wok/pluginsmanager.py > > diff --git a/src/wok/pluginsmanager.py b/src/wok/pluginsmanager.py > new file mode 100644 > index 0000000..d799f11 > --- /dev/null > +++ b/src/wok/pluginsmanager.py > @@ -0,0 +1,238 @@ > +# > +# Project Wok > +# > +# Copyright IBM Corp, 2016 > +# > +# This library is free software; you can redistribute it and/or > +# modify it under the terms of the GNU Lesser General Public > +# License as published by the Free Software Foundation; either > +# version 2.1 of the License, or (at your option) any later version. > +# > +# This library is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > +# Lesser General Public License for more details. > +# > +# You should have received a copy of the GNU Lesser General Public > +# License along with this library; if not, write to the Free Software > +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > +# > + > + > +import cherrypy > +import os > +import xml.etree.ElementTree as ET > +from cherrypy.lib.reprconf import Parser > +from configobj import ConfigObj > + > + > +from basemodel import Singleton > +from wok.config import paths, PluginConfig, PluginPaths > +from wok.exception import OperationFailed > +from wok.utils import import_class, wok_log > + > + > +class Plugins(): > + __metaclass__ = Singleton > + > + def __init__(self, options=None): > + # { '<PLUGIN_NAME>': { > + # config: <PLUGIN_CHERRYPY_CONFIG>, > + # enabled: <TRUE/FALSE>, > + # uri: <PLUGIN_URI_FROM_FILE_CONFIG>, > + # app: <PLUGIN_MAIN_MODULE> > + # conf_file: <PLUGIN_CONFIGURATION_FILE> > + self._plugins_dict = {} > + self.options = options > + self._init_all_plugins() > + > + def _load_plugin_app(self, name): > + """ > + Loads the plugin main module > + """ > + plugin_class = ('plugins.%s.%s' % (name, name[0].upper() + name[1:])) > + try: > + plugin_app = import_class(plugin_class)(self.options) > + except ImportError, e: > + wok_log.error("Failed to import plugin %s, error: %s" % > + (plugin_class, e.message)) > + self._plugins_dict[name]['enabled'] = False > + self._plugins_dict[name]['app'] = None > + return > + self._plugins_dict[name]['app'] = plugin_app > + > + def _load_plugin_app_config(self, name): > + """ > + Sets the plugin's cherrypy configuration. That is a merge between Wok > + standard configuration plus plugin's self configuration > + """ > + app_conf = {} > + app_conf.update(PluginConfig(name)) > + plugin_app = self._plugins_dict[name]['app'] > + if plugin_app is None: > + return > + > + # dynamically extend plugin config with custom data, if provided > + get_custom_conf = getattr(plugin_app, "get_custom_conf", None) > + if get_custom_conf is not None: > + app_conf.update(get_custom_conf()) > + > + # dynamically add tools.wokauth.on = True to extra plugin APIs > + try: > + sub_nodes = import_class('plugins.%s.control.sub_nodes' % > + name) > + urlSubNodes = {} > + for ident, node in sub_nodes.items(): > + if node.url_auth: > + ident = "/%s" % ident > + urlSubNodes[ident] = {'tools.wokauth.on': True} > + app_conf.update(urlSubNodes) > + except ImportError, e: > + wok_log.error("Failed to import subnodes for plugin %s, error: %s" > + % (name, e.message)) > + self._plugins_dict[name]['config'] = app_conf > + > + def _load_plugin_config_file(self, name): > + """ > + Loads the information from Wok section in <plugin>.conf file. Currently > + two tags/options are required to Wok and section looks like: > + [wok] > + enable = True > + uri = '/plugins/mypluginpath' > + """ > + plugin_conf_file = PluginPaths(name).conf_file > + config = {} > + if not os.path.exists(plugin_conf_file): > + wok_log.error("Plugin configuration file %s doesn't exist." % > + plugin_conf_file) > + else: > + try: > + config = Parser().dict_from_file(plugin_conf_file) > + except ValueError as e: > + msg = "Failed to load plugin conf from %s: %s" > + wok_log.error(msg % (plugin_conf_file, e.message)) > + > + plugin = self._plugins_dict[name] > + plugin['conf_file'] = plugin_conf_file > + plugin['enabled'] = config.get('wok', {}).get('enable', False) > + plugin['uri'] = config.get('wok', {}).get('uri', 'Unknow') > + This block should be on 'else' statement (as part of the 'try' block') or add a 'return' in the 'if' statement as the config file was not found. Otherwise, it will raise an exception as 'config' may be not defined if it does not entered on 'else' block. The same will happen if it goes to the 'except' block. So move it to 'try' block. Tip: if you add 'return' in the 'if' statement you don't need the 'else' anymore and eliminate one indentation level. > + def _init_all_plugins(self): > + """ > + Initializes internal plugin dictionary, searching all directories in > + <install_path>/wok/plugins, each directory should store a plugin > + content. Then loads its configuration file in order to set as enabled > + or disabled. > + """ > + plugin_dir = paths.plugins_dir > + try: > + dir_contents = os.listdir(plugin_dir) > + except OSError as e: > + wok_log.error("Failed to fetch plugins from '%s': %s" % > + (plugin_dir, e.message)) > + return > + for name in dir_contents: > + if os.path.isdir(os.path.join(plugin_dir, name)): > + # TODO: > + # Add command line option to disable plugin by its name > + # > + self._plugins_dict[name] = {} > + self._plugins_dict[name]['config'] = {} > + self._plugins_dict[name]['app'] = None > + self._load_plugin_config_file(name) > + > + # Disable all plugins but 'sample' in test mode > + if (self.options is not None) and self.options.test: > + self._plugins_dict[name]['enabled'] = (name == 'sample') > + Why? I could be able to see all plugins running on test mode. For example, Kimchi provides an MockModel to allow user tests Kimchi without making any change in the host system. > + if not self._plugins_dict[name]['enabled']: > + wok_log.info("Initializing plugin: %s [DISABLED]" % name) > + else: > + wok_log.info("Initializing plugin: %s [ENABLED]" % name) > + > + def _set_cherrypy_app(self, name): > + """ > + Load plugin module and cherrypy configuration, then set it as cherrypy > + app. > + """ > + self._load_plugin_app(name) > + self._load_plugin_app_config(name) > + if self._plugins_dict[name]['enabled']: > + cherrypy.tree.mount( > + self._plugins_dict[name]['app'], > + self._plugins_dict[name]['uri'], > + self._plugins_dict[name]['config']) > + wok_log.info("Plugin '%s' loaded" % > + self._plugins_dict[name]['app']) > + > + def load_plugins(self): > + """ > + Set enabled plugins into Cherrypy > + """ > + for plugin in self.get_enabled_plugins(): > + self._set_cherrypy_app(plugin) > + > + def get_all_plugins_info(self): > + return self._plugins_dict > + > + def get_plugin_info(self, name): > + return self._plugins_dict.get(name, {}) > + > + def get_all_plugins_names(self): > + ret = self._plugins_dict.keys() > + ret.sort() > + return ret > + > + def get_enabled_plugins(self): > + ret = [plugin for plugin in self._plugins_dict.keys() if > + self._plugins_dict[plugin]['enabled']] > + ret.sort() > + return ret > + > + def _enable_plugin_conf_file(self, f_conf, enable=True): > + try: > + # 'unrepr' makes ConfigObj read and write characters like ("'?) > + conf = ConfigObj(infile=f_conf, unrepr=True) > + conf['wok']['enable'] = enable > + with open(f_conf, 'wb') as f: > + conf.write(f) > + except Exception as e: > + wok_log.error('Error updating plugin conf file. ' + e.message) > + raise > + > + def _change_plugin_state(self, name, enable=True): > + plugin = self._plugins_dict[name] > + if plugin['enabled'] == enable: > + return > + try: > + self._enable_plugin_conf_file(plugin['conf_file'], enable) > + except: > + raise OperationFailed('WOKPLUG0002E', {'plugin': name}) > + plugin['enabled'] = enable > + > + def enable_plugin(self, plugin): > + wok_log.info("PluginsManager: Enabling plugin '%s'" % plugin) > + self._change_plugin_state(plugin, True) > + > + def disable_plugin(self, plugin): > + wok_log.info("PluginsManager: Disabling plugin '%s'" % plugin) > + self._change_plugin_state(plugin, False) > + > + > +def get_all_tabs(): > + files = [] > + > + for plugin in Plugins().get_enabled_plugins(): > + files.append(os.path.join(PluginPaths(plugin).ui_dir, > + 'config/tab-ext.xml')) > + > + tabs = [] > + for f in files: > + try: > + root = ET.parse(f) > + except (IOError): > + wok_log.debug("Unable to load %s", f) > + continue > + tabs.extend([t.text.lower() for t in root.getiterator('title')]) > + > + return tabs

2 6

Re: [Kimchi-devel] [PATCH v2][Wok 8/8] Add Plugins Manager test cases
by Aline Manera 08 Jun '16

08 Jun '16

Unless the idea to only have sample plugin enabled on test mode, this patch looks good for me. On 06/06/2016 04:13 PM, Rodrigo Trujillo wrote: > Test cases for new API > > Signed-off-by: Rodrigo Trujillo <rodrigo.trujillo(a)linux.vnet.ibm.com> > --- > tests/test_plugin.py | 103 +++++++++++++++++++++++++++++++++++++++++++++++++-- > 1 file changed, 99 insertions(+), 4 deletions(-) > > diff --git a/tests/test_plugin.py b/tests/test_plugin.py > index d785cfa..01a3be3 100644 > --- a/tests/test_plugin.py > +++ b/tests/test_plugin.py > @@ -20,11 +20,13 @@ > # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > > import json > +import os > import unittest > from functools import partial > > -#from wok.pluginsmanager import get_enabled_plugins > +from wok.config import paths > from wok.pluginsmanager import Plugins > +from wok.utils import wok_log > > import utils > > @@ -50,9 +52,9 @@ def tearDownModule(): > test_server.stop() > > > -(a)unittest.skipUnless( > - 'sample' in Plugins().get_enabled_plugins(), > - 'sample plugin is not enabled, skip this test!') > +# > +# In test mode, the plugin Sample should always be enabled > +# > class PluginTests(unittest.TestCase): > > def setUp(self): > @@ -121,3 +123,96 @@ class PluginTests(unittest.TestCase): > req = json.dumps({'name': 'nowidth', 'length': 40}) > resp = self.request('/plugins/sample/rectangles', req, 'POST') > self.assertEquals(400, resp.status) > + > + > +def _plugins_number(): > + """ Find number of plugins in <wokpath>/wok/plugins""" > + plugin_dir = paths.plugins_dir > + try: > + return len([d for d in os.listdir(plugin_dir) > + if os.path.isdir(os.path.join(plugin_dir, d))]) > + except OSError as e: > + wok_log.error("Failed reading plugins directory '%s': %s" % > + (plugin_dir, e.message)) > + return 0 > + > + > +class PluginsManagerTests(unittest.TestCase): > + """ > + This class test functionalities of Plugins Manager and plugins API, using > + the Sample plugin, which should be enabled by default during tests > + """ > + > + def setUp(self): > + self.request = partial(utils.request, host, ssl_port) > + self.info = Plugins().get_plugin_info('sample') > + with open(self.info['conf_file']) as f: > + self.enable_sample_plugin = ('enable = True' in f.read()) > + > + def test_api(self): > + # Only SAMPLE should be enabled > + out = [{'name': 'sample', 'enabled': True}] > + > + # List of plugins > + resp = self.request('/pluginsmanager', {}, 'GET') > + plugins = json.loads(resp.read()) > + self.assertEquals(_plugins_number(), len(plugins)) > + self.assertIn(out[0], plugins) > + > + # Wrong plugin name > + resp = self.request('/pluginsmanager/samplewrong', {}, 'GET') > + self.assertEquals(404, resp.status) > + self.assertIn('WOKPLUG0001E', json.loads(resp.read())['reason']) > + > + # Lookup plugin name > + resp = self.request('/pluginsmanager/sample', {}, 'GET') > + self.assertEquals(200, resp.status) > + self.assertDictEqual(out[0], json.loads(resp.read())) > + > + # Enabling wrong plugin > + resp = self.request('/pluginsmanager/samplewrong/enable', {}, 'POST') > + self.assertEquals(404, resp.status) > + self.assertIn('WOKPLUG0001E', json.loads(resp.read())['reason']) > + > + # Enabling Sample (already enabled, no problem) > + resp = self.request('/pluginsmanager/sample/enable', {}, 'POST') > + self.assertEquals(200, resp.status) > + self.assertDictEqual(out[0], json.loads(resp.read())) > + > + # Disabling Sample > + resp = self.request('/pluginsmanager/sample/disable', {}, 'POST') > + self.assertEquals(200, resp.status) > + out[0]['enabled'] = False > + self.assertDictEqual(out[0], json.loads(resp.read())) > + > + # Disabling Sample (already disabled, no problem) > + resp = self.request('/pluginsmanager/sample/disable', {}, 'POST') > + self.assertEquals(200, resp.status) > + self.assertDictEqual(out[0], json.loads(resp.read())) > + > + # Check config file > + with open(self.info['conf_file']) as f: > + self.assertIn('enable = False', f.read()) > + > + # Backing configuration file to initial state > + if self.enable_sample_plugin: > + Plugins()._enable_plugin_conf_file(self.info['conf_file']) > + > + def test_not_allowed_methods(self): > + # Only GET allowed in Collection > + resp = self.request('/pluginsmanager', {}, 'POST') > + self.assertEquals(405, resp.status) > + resp = self.request('/pluginsmanager', {}, 'PUT') > + self.assertEquals(405, resp.status) > + resp = self.request('/pluginsmanager', {}, 'DELETE') > + self.assertEquals(405, resp.status) > + > + # Only GET/POST allowed in resource > + resp = self.request('/pluginsmanager/sample', {}, 'PUT') > + self.assertEquals(405, resp.status) > + resp = self.request('/pluginsmanager/sample', {}, 'DELETE') > + self.assertEquals(405, resp.status) > + resp = self.request('/pluginsmanager/sample/enable', {}, 'PUT') > + self.assertEquals(405, resp.status) > + resp = self.request('/pluginsmanager/sample/disable', {}, 'DELETE') > + self.assertEquals(405, resp.status)

2 4