April 2014 - Kimchi-devel - oVirt List Archives

[PATCH] kimchi.exception: Properly Decode All Kinds of Exception Arguments
by zhshzhou＠linux.vnet.ibm.com 03 Apr '14

03 Apr '14

From: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> KimchiException provides the ability to translate each argument and the error message. It decodes the translated message to unicode. It's also smart and avoids decoding a unicode argument twice by a check to make sure the to-be-decoded value is not unicode. The problem is that when a KimchiException is initialized by another KimchiException, for example, try: ... except OperationFailed as e: ... raise OperationFailed('KCHXXXX', {'err': e}) the variable "e" is not a unicode object so it passes the check mentioned above, then it gets decoded twice and raises an encoding exception. This patch does not only rely on the type of the to-be-decoded object, but also catches the encoding exeption. It firstly try to call str(obj) to get the ascii string of the obj, then decodes it to unicode. This is valid for a normal string or any object can format itself to a normal string. If it fails, it try to call unicode(obj) to get the unicode string of the object. This is valid for objects can format themselves to a unicode string. In all, it handles various objects like following. If obj is a unicode string, use it directly. If obj is a normal string, decode it to a unicode string. If obj can format itself to a normal string, format it and decode it to unicode. If obj can format itself to a unicode string, format it. Another problem is that by the current design, KimchiException always stores error message in unicode. This can cause encoding exception when we call "str(KimchiExceptionObject)". This patch is not a total refactor of the translation and encoding so it does not solve this problem. The workaround is using KimchiExceptionObject.message instead of str it. Notice: I'm following the Python str.encode and decode semantics in the commit message. "encode" means converting unicode sting to ascii/utf-8. "decode" means converting ascii/utf-8 string to unicode. "encoding" means the action of decode and encode in general. Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- src/kimchi/asynctask.py | 2 +- src/kimchi/exception.py | 8 +++++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/src/kimchi/asynctask.py b/src/kimchi/asynctask.py index 8f0d96c..54fb749 100644 --- a/src/kimchi/asynctask.py +++ b/src/kimchi/asynctask.py @@ -68,4 +68,4 @@ class AsyncTask(object): except Exception, e: cherrypy.log.error_log.error("Error in async_task %s " % self.id) cherrypy.log.error_log.error(traceback.format_exc()) - cb("Unexpected exception: %s" % str(e), False) + cb("Unexpected exception: %s" % e.message, False) diff --git a/src/kimchi/exception.py b/src/kimchi/exception.py index 87982ea..fcf60cc 100644 --- a/src/kimchi/exception.py +++ b/src/kimchi/exception.py @@ -54,7 +54,13 @@ class KimchiException(Exception): for key, value in args.iteritems(): if not isinstance(value, unicode): - args[key] = unicode(str(value), 'utf-8') + try: + # In case the value formats itself to an ascii string. + args[key] = unicode(str(value), 'utf-8') + except UnicodeEncodeError: + # In case the value is a KimchiException or it formats + # itself to a unicode string. + args[key] = unicode(value) return unicode(translation.gettext(text), 'utf-8') % args -- 1.8.5.3

4 4

[PATCH V2] add a make check-local command to verify the i18n string format
by shaohef＠linux.vnet.ibm.com 03 Apr '14

03 Apr '14

From: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> V2 -> V3: improve the regular expression pattern. V1 -> V2: move the check to top makefile. check all i18n.py improve the checking information. ShaoHe Feng (1): add a make check-local command to verify the i18n string format Makefile.am | 8 ++++++++ 1 file changed, 8 insertions(+) -- 1.8.5.3

4 5

[PATCH V4 0/6] Fix 'disk full' issue
by Rodrigo Trujillo 02 Apr '14

02 Apr '14

V4: - Fix pep8 issues V3: - Fix issues with tests - Rebase with 1.2 V2: Address Aline's comments: - Change error message - Aggregate 'if's V1: If the disk where objectstore (Kimchi database) is full, then a lot of errors will be raised without any special treatment. This can lead the system to an unexpected state. This patchset modifies kimchi in order to give the right treatment to exceptions, showing the error to the user or hidding when possible. Rodrigo Trujillo (6): Fix 'disk full' issue: Change objectstore exception handling Fix 'disk full' issue: Fix Templates db store/delete error handling Fix 'disk full' issue: Fix storage volume error handling Fix 'disk full' issue: Fix storagepool and asynctasks error handling Fix 'disk full' issue: Fix vms/screenshot db store/delete error handling Fix PEP8 minor issues in multiple files src/kimchi/asynctask.py | 7 ++-- src/kimchi/control/host.py | 1 + src/kimchi/i18n.py | 5 +++ src/kimchi/model/storagepools.py | 7 ++-- src/kimchi/model/storagevolumes.py | 49 +++++++++++++++++---------- src/kimchi/model/templates.py | 27 +++++++++++---- src/kimchi/model/vms.py | 69 +++++++++++++++++++++++++++----------- src/kimchi/objectstore.py | 7 ++++ 8 files changed, 124 insertions(+), 48 deletions(-) -- 1.8.5.3

2 7

[PATCH V3 0/5] Fix 'disk full' issue
by Rodrigo Trujillo 02 Apr '14

02 Apr '14

V3: - Fix issues with tests - Rebase with 1.2 V2: Address Aline's comments: - Change error message - Aggregate 'if's V1: If the disk where objectstore (Kimchi database) is full, then a lot of errors will be raised without any special treatment. This can lead the system to an unexpected state. This patchset modifies kimchi in order to give the right treatment to exceptions, showing the error to the user or hidding when possible. Rodrigo Trujillo (5): Fix 'disk full' issue: Change objectstore exception handling Fix 'disk full' issue: Fix Templates db store/delete error handling Fix 'disk full' issue: Fix storage volume error handling Fix 'disk full' issue: Fix storagepool and asynctasks error handling Fix 'disk full' issue: Fix vms/screenshot db store/delete error handling src/kimchi/asynctask.py | 7 ++-- src/kimchi/i18n.py | 5 +++ src/kimchi/model/storagepools.py | 8 +++-- src/kimchi/model/storagevolumes.py | 49 ++++++++++++++++---------- src/kimchi/model/templates.py | 27 +++++++++++---- src/kimchi/model/vms.py | 70 +++++++++++++++++++++++++++----------- src/kimchi/objectstore.py | 7 ++++ 7 files changed, 123 insertions(+), 50 deletions(-) -- 1.8.5.3

3 11

[PATCH v2] Add the command to run_command error and debug logging
by Christy Perez 02 Apr '14

02 Apr '14

I was debugging an issue and seeing rc's but not completely sure which command they were caused by. I think it's just nice to include what generated an rc, and also have all the output on the same line to remove any uncertainty. For example, instead of: rc: -15 error: rc: -15 error: You'll see: rc: -15 error: returned from cmd: /usr/bin/example-command-here Signed-off-by: Christy Perez <christy(a)linux.vnet.ibm.com> --- src/kimchi/utils.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/kimchi/utils.py b/src/kimchi/utils.py index 6c29e0e..a30dcfe 100644 --- a/src/kimchi/utils.py +++ b/src/kimchi/utils.py @@ -174,9 +174,11 @@ def run_command(cmd, timeout=None): kimchi_log.debug("out:\n%s", out) if proc.returncode != 0: - kimchi_log.error("rc: %s\nerror:\n%s", proc.returncode, error) + kimchi_log.error("rc: %s error: %s returned from cmd: %s", + proc.returncode, error, ' '.join(cmd)) elif error: - kimchi_log.debug("error:\n%s", error) + kimchi_log.debug("error: %s returned from cmd: %s", + error, ' '.join(cmd)) if timeout_flag[0]: msg = ("subprocess is killed by signal.SIGKILL for " -- 1.8.5.3

2 1

[PATCH] Bug #361. Network Action Dropdown does not Disappear when mouse out
by huoyuxin＠linux.vnet.ibm.com 02 Apr '14

02 Apr '14

From: Yu Xin Huo <huoyuxin(a)linux.vnet.ibm.com> Signed-off-by: Yu Xin Huo <huoyuxin(a)linux.vnet.ibm.com> --- ui/js/src/kimchi.network.js | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/ui/js/src/kimchi.network.js b/ui/js/src/kimchi.network.js index 1642b99..9fa8324 100644 --- a/ui/js/src/kimchi.network.js +++ b/ui/js/src/kimchi.network.js @@ -149,6 +149,9 @@ kimchi.addNetworkActions = function(network) { }).click(function() { $(".menu-container", "#" + network.name).toggle(); window.scrollBy(0, 150); + }).mouseleave(function() { + var delay = setTimeout(function(){$(".menu-container", "#"+network.name).toggle(false);}, 500); + $(".menu-container", "#"+network.name).mouseenter(function(){clearTimeout(delay);}); }); $(".menu-container", "#" + network.name).mouseleave(function() { $(".menu-container", "#" + network.name).toggle(false); -- 1.7.1

3 2

[PATCH V2] bug fix: we should pass str to cherrpy HTTPErorr
by shaohef＠linux.vnet.ibm.com 02 Apr '14

02 Apr '14

From: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> The cherrpy HTTPError accepts message as string, it will generate a error page, and encode this whole page. ERROR:cherrypy.error:[01/Apr/2014:17:46:28] ENGINE Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/cherrypy/wsgiserver/wsgiserver2.py", line 1302, in communicate req.respond() File "/usr/lib/python2.7/site-packages/cherrypy/wsgiserver/wsgiserver2.py", line 831, in respond self.server.gateway(self).respond() File "/usr/lib/python2.7/site-packages/cherrypy/wsgiserver/wsgiserver2.py", line 2126, in respond chunk = chunk.encode('ISO-8859-1') UnicodeEncodeError: 'latin-1' codec can't encode characters in position 529-536: ordinal not in range(256) Signed-off-by: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> --- src/kimchi/auth.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/kimchi/auth.py b/src/kimchi/auth.py index 3b33d7f..63e11c4 100644 --- a/src/kimchi/auth.py +++ b/src/kimchi/auth.py @@ -238,4 +238,4 @@ def kimchiauth(admin_methods=None): cherrypy.response.headers['WWW-Authenticate'] = 'Basic realm=kimchi' e = InvalidOperation('KCHAUTH0002E') - raise cherrypy.HTTPError(401, e.message) + raise cherrypy.HTTPError(401, e.message.encode('utf-8')) -- 1.8.5.3

2 2

[PATCH] Issue#343 & #353: Improve&Correct UI Init Logic Flow
by Hongliang Wang 02 Apr '14

02 Apr '14

Fixed the 2 issues: 1) When refreshing (F5) a plugin tab, default tab will be redirected then; 2) tabs.xml is requested twice. Signed-off-by: Hongliang Wang <hlwang(a)linux.vnet.ibm.com> --- ui/js/src/kimchi.main.js | 127 +++++++++++++++++++++++++++-------------------- 1 file changed, 72 insertions(+), 55 deletions(-) diff --git a/ui/js/src/kimchi.main.js b/ui/js/src/kimchi.main.js index aac7e28..9b0acbf 100644 --- a/ui/js/src/kimchi.main.js +++ b/ui/js/src/kimchi.main.js @@ -16,10 +16,75 @@ * limitations under the License. */ kimchi.main = function() { - var tabUrl = "/config/ui/tabs.xml"; - var DEFAULT_HASH = kimchi.getDefaultPage(tabUrl); kimchi.popable(); + var genTabs = function(tabs) { + var tabsHtml = []; + $(tabs).each(function(i, tab) { + var title = tab['title']; + var path = tab['path']; + tabsHtml.push( + '<li>', + '<a class="item" href="', path, '">', + title, + '</a>', + '</li>' + ); + }); + return tabsHtml.join(''); + }; + + var parseTabs = function(xmlData) { + var tabs = []; + $(xmlData).find('tab').each(function() { + var $tab = $(this); + var titleKey = $tab.find('title').text(); + var title = i18n[titleKey]; + var path = $tab.find('path').text(); + tabs.push({ + title: title, + path: path + }); + }); + + return tabs; + }; + + var retrieveTabs = function(url) { + var tabs; + $.ajax({ + url : url, + async : false, + success : function(xmlData) { + tabs = parseTabs(xmlData); + } + }); + return tabs; + }; + + var tabConfigUrl = '/config/ui/tabs.xml'; + var pluginConfigUrl = '/plugins/{plugin}/ui/config/tab-ext.xml'; + var DEFAULT_HASH; + var buildTabs = function(callback) { + var tabs = retrieveTabs(tabConfigUrl); + kimchi.listPlugins(function(plugins) { + $(plugins).each(function(i, p) { + var url = kimchi.template(pluginConfigUrl, { + plugin: p + }); + tabs.concat(retrieveTabs(url)); + }); + + var firstTabPath = tabs[0] && tabs[0]['path']; + DEFAULT_HASH = firstTabPath && + firstTabPath.substring(0, firstTabPath.length - 5); + + $('#nav-menu').append(genTabs(tabs)); + + callback && callback(); + }); + }; + var onLanguageChanged = function(lang) { kimchi.lang.set(lang); location.reload(); @@ -132,8 +197,6 @@ kimchi.main = function() { }); // Perform logging out via Ajax request. - - $('#btn-logout').on('click', function() { kimchi.logout(function() { updatePage(); @@ -145,9 +208,7 @@ kimchi.main = function() { $('#btn-help').on('click', kimchi.getHelp); }; - // Load i18n translation strings first and then render the page. - $('#main').load('i18n.html', function() { - kimchi.addTabs(tabUrl); + var initUI = function() { $(document).bind('ajaxError', function(event, jqXHR, ajaxSettings, errorThrown) { if (!ajaxSettings['kimchi']) { return; @@ -175,56 +236,12 @@ kimchi.main = function() { kimchi.user.showUser(true); initListeners(); updatePage(); - }); -}; - -kimchi.addTabs = function(url) { - var tabsHtml = kimchi.getTabHtml(url); - $('#nav-menu').prepend(tabsHtml); - kimchi.addExtTabs(); -}; - -kimchi.addExtTabs = function() { - kimchi.listPlugins(function(results) { - for ( var i = 0; i < results.length; i++) { - var tabsHtml = kimchi.getTabHtml("/plugins/" + results[i] + "/ui/config/tab-ext.xml"); - $('#nav-menu').append(tabsHtml); - } - }); -}; - -kimchi.getDefaultPage = function(url) { - var defautLocation = ""; - $.ajax({ - url : url, - async : false, - success : function(xmlData) { - var tab = $(xmlData).find('tab').first(); - var path = tab.find('path').text(); - if (path) { - defautLocation = path.substring(0, path.length - 5); - } - } - }); - return defautLocation; -}; + }; -kimchi.getTabHtml = function(url) { - var tabsHtml = ""; - $.ajax({ - url : url, - async : false, - success : function(xmlData) { - $(xmlData).find('tab').each(function() { - var $tab = $(this); - var titleKey = $tab.find('title').text(); - var title = i18n[titleKey]; - var path = $tab.find('path').text(); - tabsHtml += "<li><a class='item' href=" + path + ">" + title + "</a></li>"; - }); - } + // Load i18n translation strings first and then render the page. + $('#main').load('i18n.html', function() { + buildTabs(initUI); }); - return tabsHtml; }; kimchi.getHelp = function(e) { -- 1.8.1.4

2 2

[PATCH] AsyncTask: Propagate cherrypy request information to worker threads
by zhshzhou＠linux.vnet.ibm.com 02 Apr '14

02 Apr '14

From: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> cherrypy.request is a proxy for cherrypy.serving.request, and cherrypy.serving stores the request information on the thread local storage. When new threads created by Kimchi code, the related request information is not propagated to the new threads. The problem is that exception message translation code needs the Accept-Language header or session cookie to determine which language the user speaks. If the request information is missing, it assumes the language is English. Usually this is not a problem because most of the threads in Kimchi just run in background, and the exception message is written to log, so English in log is OK. However the AsyncTask catches the exception in thread and reports to the front-end, and this is not friendly to the user without proper translation. This patch propagates the request information to the new threads started by the AsyncTask, so exception message gets translated according to the request header or session cookie. Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- src/kimchi/asynctask.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/kimchi/asynctask.py b/src/kimchi/asynctask.py index 8f0d96c..b54f143 100644 --- a/src/kimchi/asynctask.py +++ b/src/kimchi/asynctask.py @@ -37,6 +37,7 @@ class AsyncTask(object): self.status = 'running' self.message = 'OK' self._save_helper() + self._cp_request = cherrypy.serving.request self.thread = threading.Thread(target=self._run_helper, args=(opaque, self._status_cb)) self.thread.setDaemon(True) @@ -63,6 +64,7 @@ class AsyncTask(object): session.store('task', self.id, obj) def _run_helper(self, opaque, cb): + cherrypy.serving.request = self._cp_request try: self.fn(cb, opaque) except Exception, e: -- 1.8.5.3

3 5

[PATCH] UI: change some code of storage add page
by shaohef＠linux.vnet.ibm.com 02 Apr '14

02 Apr '14

From: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> Actually there is no bug. But we think, we can improve the codes. We can remove the reduplicate codes. Signed-off-by: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- ui/js/src/kimchi.storagepool_add_main.js | 42 +++++++++----------------------- 1 file changed, 11 insertions(+), 31 deletions(-) diff --git a/ui/js/src/kimchi.storagepool_add_main.js b/ui/js/src/kimchi.storagepool_add_main.js index ea9f68b..86dbe7f 100644 --- a/ui/js/src/kimchi.storagepool_add_main.js +++ b/ui/js/src/kimchi.storagepool_add_main.js @@ -123,37 +123,17 @@ kimchi.initStorageAddPage = function() { }); $('#poolTypeInputId').change(function() { - if ($(this).val() === 'dir') { - $('.path-section').removeClass('tmpl-html'); - $('.logical-section').addClass('tmpl-html'); - $('.nfs-section').addClass('tmpl-html'); - $('.iscsi-section').addClass('tmpl-html'); - $('.scsi-section').addClass('tmpl-html'); - } else if ($(this).val() === 'netfs') { - $('.path-section').addClass('tmpl-html'); - $('.logical-section').addClass('tmpl-html'); - $('.nfs-section').removeClass('tmpl-html'); - $('.iscsi-section').addClass('tmpl-html'); - $('.scsi-section').addClass('tmpl-html'); - } else if ($(this).val() === 'iscsi') { - $('.path-section').addClass('tmpl-html'); - $('.logical-section').addClass('tmpl-html'); - $('.nfs-section').addClass('tmpl-html'); - $('.iscsi-section').removeClass('tmpl-html'); - $('.scsi-section').addClass('tmpl-html'); - } else if ($(this).val() === 'logical') { - $('.path-section').addClass('tmpl-html'); - $('.logical-section').removeClass('tmpl-html'); - $('.nfs-section').addClass('tmpl-html'); - $('.iscsi-section').addClass('tmpl-html'); - $('.scsi-section').addClass('tmpl-html'); - } else if ($(this).val() === 'scsi') { - $('.scsi-section').removeClass('tmpl-html'); - $('.path-section').addClass('tmpl-html'); - $('.logical-section').addClass('tmpl-html'); - $('.nfs-section').addClass('tmpl-html'); - $('.iscsi-section').addClass('tmpl-html'); - } + var poolObject = {'dir': ".path-section", 'netfs': '.nfs-section', + 'iscsi': '.iscsi-section', 'scsi': '.scsi-section', + 'logical': '.logical-section'} + var selectType = $(this).val(); + $.each(poolObject, function(type, value) { + if(selectType == type){ + $(value).removeClass('tmpl-html'); + } else { + $(value).addClass('tmpl-html'); + } + }); }); $('#authId').click(function() { if ($(this).prop("checked")) { -- 1.8.5.3

4 4