[PATCH] kimchi.exception: Properly Decode All Kinds of Exception Arguments
From: Zhou Zheng Sheng <zhshzhou@linux.vnet.ibm.com> KimchiException provides the ability to translate each argument and the error message. It decodes the translated message to unicode. It's also smart and avoids decoding a unicode argument twice by a check to make sure the to-be-decoded value is not unicode. The problem is that when a KimchiException is initialized by another KimchiException, for example, try: ... except OperationFailed as e: ... raise OperationFailed('KCHXXXX', {'err': e}) the variable "e" is not a unicode object so it passes the check mentioned above, then it gets decoded twice and raises an encoding exception. This patch does not only rely on the type of the to-be-decoded object, but also catches the encoding exeption. It firstly try to call str(obj) to get the ascii string of the obj, then decodes it to unicode. This is valid for a normal string or any object can format itself to a normal string. If it fails, it try to call unicode(obj) to get the unicode string of the object. This is valid for objects can format themselves to a unicode string. In all, it handles various objects like following. If obj is a unicode string, use it directly. If obj is a normal string, decode it to a unicode string. If obj can format itself to a normal string, format it and decode it to unicode. If obj can format itself to a unicode string, format it. Another problem is that by the current design, KimchiException always stores error message in unicode. This can cause encoding exception when we call "str(KimchiExceptionObject)". This patch is not a total refactor of the translation and encoding so it does not solve this problem. The workaround is using KimchiExceptionObject.message instead of str it. Notice: I'm following the Python str.encode and decode semantics in the commit message. "encode" means converting unicode sting to ascii/utf-8. "decode" means converting ascii/utf-8 string to unicode. "encoding" means the action of decode and encode in general. Signed-off-by: Zhou Zheng Sheng <zhshzhou@linux.vnet.ibm.com> --- src/kimchi/asynctask.py | 2 +- src/kimchi/exception.py | 8 +++++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/src/kimchi/asynctask.py b/src/kimchi/asynctask.py index 8f0d96c..54fb749 100644 --- a/src/kimchi/asynctask.py +++ b/src/kimchi/asynctask.py @@ -68,4 +68,4 @@ class AsyncTask(object): except Exception, e: cherrypy.log.error_log.error("Error in async_task %s " % self.id) cherrypy.log.error_log.error(traceback.format_exc()) - cb("Unexpected exception: %s" % str(e), False) + cb("Unexpected exception: %s" % e.message, False) diff --git a/src/kimchi/exception.py b/src/kimchi/exception.py index 87982ea..fcf60cc 100644 --- a/src/kimchi/exception.py +++ b/src/kimchi/exception.py @@ -54,7 +54,13 @@ class KimchiException(Exception): for key, value in args.iteritems(): if not isinstance(value, unicode): - args[key] = unicode(str(value), 'utf-8') + try: + # In case the value formats itself to an ascii string. + args[key] = unicode(str(value), 'utf-8') + except UnicodeEncodeError: + # In case the value is a KimchiException or it formats + # itself to a unicode string. + args[key] = unicode(value) return unicode(translation.gettext(text), 'utf-8') % args -- 1.8.5.3
Hi Aline, This patch fix the bug triggered by "AsyncTask: Propagate cherrypy request information to worker threads". on 2014/04/02 13:42, zhshzhou@linux.vnet.ibm.com wrote:
From: Zhou Zheng Sheng <zhshzhou@linux.vnet.ibm.com>
KimchiException provides the ability to translate each argument and the error message. It decodes the translated message to unicode. It's also smart and avoids decoding a unicode argument twice by a check to make sure the to-be-decoded value is not unicode.
The problem is that when a KimchiException is initialized by another KimchiException, for example,
try: ... except OperationFailed as e: ... raise OperationFailed('KCHXXXX', {'err': e})
the variable "e" is not a unicode object so it passes the check mentioned above, then it gets decoded twice and raises an encoding exception.
This patch does not only rely on the type of the to-be-decoded object, but also catches the encoding exeption. It firstly try to call str(obj) to get the ascii string of the obj, then decodes it to unicode. This is valid for a normal string or any object can format itself to a normal string. If it fails, it try to call unicode(obj) to get the unicode string of the object. This is valid for objects can format themselves to a unicode string. In all, it handles various objects like following.
If obj is a unicode string, use it directly. If obj is a normal string, decode it to a unicode string. If obj can format itself to a normal string, format it and decode it to unicode. If obj can format itself to a unicode string, format it.
Another problem is that by the current design, KimchiException always stores error message in unicode. This can cause encoding exception when we call "str(KimchiExceptionObject)". This patch is not a total refactor of the translation and encoding so it does not solve this problem. The workaround is using KimchiExceptionObject.message instead of str it.
Notice: I'm following the Python str.encode and decode semantics in the commit message. "encode" means converting unicode sting to ascii/utf-8. "decode" means converting ascii/utf-8 string to unicode. "encoding" means the action of decode and encode in general.
Signed-off-by: Zhou Zheng Sheng <zhshzhou@linux.vnet.ibm.com> --- src/kimchi/asynctask.py | 2 +- src/kimchi/exception.py | 8 +++++++- 2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/src/kimchi/asynctask.py b/src/kimchi/asynctask.py index 8f0d96c..54fb749 100644 --- a/src/kimchi/asynctask.py +++ b/src/kimchi/asynctask.py @@ -68,4 +68,4 @@ class AsyncTask(object): except Exception, e: cherrypy.log.error_log.error("Error in async_task %s " % self.id) cherrypy.log.error_log.error(traceback.format_exc()) - cb("Unexpected exception: %s" % str(e), False) + cb("Unexpected exception: %s" % e.message, False) diff --git a/src/kimchi/exception.py b/src/kimchi/exception.py index 87982ea..fcf60cc 100644 --- a/src/kimchi/exception.py +++ b/src/kimchi/exception.py @@ -54,7 +54,13 @@ class KimchiException(Exception):
for key, value in args.iteritems(): if not isinstance(value, unicode): - args[key] = unicode(str(value), 'utf-8') + try: + # In case the value formats itself to an ascii string. + args[key] = unicode(str(value), 'utf-8') + except UnicodeEncodeError: + # In case the value is a KimchiException or it formats + # itself to a unicode string. + args[key] = unicode(value)
return unicode(translation.gettext(text), 'utf-8') % args
-- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshzhou@linux.vnet.ibm.com Telephone: 86-10-82454397
Reviewed-by: ShaoHe Feng <shaohef@linux.vnet.ibm.com> This patch will solve these two case. In [23]: unicode(str(iss), "utf-8") Out[23]: u'\u4f60\u597d' In [24]: class ou(object): def __str__(self): return u"你好" ....: In [25]: iu = ou() In [26]: unicode(iu) Out[26]: u'\u4f60\u597d' In [27]: class os(object): def __str__(self): return "你好" ....: In [28]: istr = os() In [29]: unicode(str(istr), "utf-8") Out[29]: u'\u4f60\u597d' On 04/02/2014 01:42 PM, zhshzhou@linux.vnet.ibm.com wrote:
From: Zhou Zheng Sheng <zhshzhou@linux.vnet.ibm.com>
KimchiException provides the ability to translate each argument and the error message. It decodes the translated message to unicode. It's also smart and avoids decoding a unicode argument twice by a check to make sure the to-be-decoded value is not unicode.
The problem is that when a KimchiException is initialized by another KimchiException, for example,
try: ... except OperationFailed as e: ... raise OperationFailed('KCHXXXX', {'err': e})
the variable "e" is not a unicode object so it passes the check mentioned above, then it gets decoded twice and raises an encoding exception.
This patch does not only rely on the type of the to-be-decoded object, but also catches the encoding exeption. It firstly try to call str(obj) to get the ascii string of the obj, then decodes it to unicode. This is valid for a normal string or any object can format itself to a normal string. If it fails, it try to call unicode(obj) to get the unicode string of the object. This is valid for objects can format themselves to a unicode string. In all, it handles various objects like following.
If obj is a unicode string, use it directly. If obj is a normal string, decode it to a unicode string. If obj can format itself to a normal string, format it and decode it to unicode. If obj can format itself to a unicode string, format it.
Another problem is that by the current design, KimchiException always stores error message in unicode. This can cause encoding exception when we call "str(KimchiExceptionObject)". This patch is not a total refactor of the translation and encoding so it does not solve this problem. The workaround is using KimchiExceptionObject.message instead of str it.
Notice: I'm following the Python str.encode and decode semantics in the commit message. "encode" means converting unicode sting to ascii/utf-8. "decode" means converting ascii/utf-8 string to unicode. "encoding" means the action of decode and encode in general.
Signed-off-by: Zhou Zheng Sheng <zhshzhou@linux.vnet.ibm.com> --- src/kimchi/asynctask.py | 2 +- src/kimchi/exception.py | 8 +++++++- 2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/src/kimchi/asynctask.py b/src/kimchi/asynctask.py index 8f0d96c..54fb749 100644 --- a/src/kimchi/asynctask.py +++ b/src/kimchi/asynctask.py @@ -68,4 +68,4 @@ class AsyncTask(object): except Exception, e: cherrypy.log.error_log.error("Error in async_task %s " % self.id) cherrypy.log.error_log.error(traceback.format_exc()) - cb("Unexpected exception: %s" % str(e), False) + cb("Unexpected exception: %s" % e.message, False) diff --git a/src/kimchi/exception.py b/src/kimchi/exception.py index 87982ea..fcf60cc 100644 --- a/src/kimchi/exception.py +++ b/src/kimchi/exception.py @@ -54,7 +54,13 @@ class KimchiException(Exception):
for key, value in args.iteritems(): if not isinstance(value, unicode): - args[key] = unicode(str(value), 'utf-8') + try: + # In case the value formats itself to an ascii string. + args[key] = unicode(str(value), 'utf-8') + except UnicodeEncodeError: + # In case the value is a KimchiException or it formats + # itself to a unicode string. + args[key] = unicode(value)
return unicode(translation.gettext(text), 'utf-8') % args
-- Thanks and best regards! Sheldon Feng(冯少合)<shaohef@linux.vnet.ibm.com> IBM Linux Technology Center
Regarding the root cause, I don't think we should display an error message with 2 exception KCHDR0005E: Unable to generate debug report my-debugreport. Details: KCHDR0003E: Unable to create debug report my-debugreport. Details: -9. It is odd. We should only raise one exception with all the details. On 04/02/2014 02:42 AM, zhshzhou@linux.vnet.ibm.com wrote:
From: Zhou Zheng Sheng <zhshzhou@linux.vnet.ibm.com>
KimchiException provides the ability to translate each argument and the error message. It decodes the translated message to unicode. It's also smart and avoids decoding a unicode argument twice by a check to make sure the to-be-decoded value is not unicode.
The problem is that when a KimchiException is initialized by another KimchiException, for example,
try: ... except OperationFailed as e: ... raise OperationFailed('KCHXXXX', {'err': e})
the variable "e" is not a unicode object so it passes the check mentioned above, then it gets decoded twice and raises an encoding exception.
This patch does not only rely on the type of the to-be-decoded object, but also catches the encoding exeption. It firstly try to call str(obj) to get the ascii string of the obj, then decodes it to unicode. This is valid for a normal string or any object can format itself to a normal string. If it fails, it try to call unicode(obj) to get the unicode string of the object. This is valid for objects can format themselves to a unicode string. In all, it handles various objects like following.
If obj is a unicode string, use it directly. If obj is a normal string, decode it to a unicode string. If obj can format itself to a normal string, format it and decode it to unicode. If obj can format itself to a unicode string, format it.
Another problem is that by the current design, KimchiException always stores error message in unicode. This can cause encoding exception when we call "str(KimchiExceptionObject)". This patch is not a total refactor of the translation and encoding so it does not solve this problem. The workaround is using KimchiExceptionObject.message instead of str it.
Notice: I'm following the Python str.encode and decode semantics in the commit message. "encode" means converting unicode sting to ascii/utf-8. "decode" means converting ascii/utf-8 string to unicode. "encoding" means the action of decode and encode in general.
Signed-off-by: Zhou Zheng Sheng <zhshzhou@linux.vnet.ibm.com> --- src/kimchi/asynctask.py | 2 +- src/kimchi/exception.py | 8 +++++++- 2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/src/kimchi/asynctask.py b/src/kimchi/asynctask.py index 8f0d96c..54fb749 100644 --- a/src/kimchi/asynctask.py +++ b/src/kimchi/asynctask.py @@ -68,4 +68,4 @@ class AsyncTask(object): except Exception, e: cherrypy.log.error_log.error("Error in async_task %s " % self.id) cherrypy.log.error_log.error(traceback.format_exc()) - cb("Unexpected exception: %s" % str(e), False) + cb("Unexpected exception: %s" % e.message, False) diff --git a/src/kimchi/exception.py b/src/kimchi/exception.py index 87982ea..fcf60cc 100644 --- a/src/kimchi/exception.py +++ b/src/kimchi/exception.py @@ -54,7 +54,13 @@ class KimchiException(Exception):
for key, value in args.iteritems(): if not isinstance(value, unicode): - args[key] = unicode(str(value), 'utf-8') + try: + # In case the value formats itself to an ascii string. + args[key] = unicode(str(value), 'utf-8') + except UnicodeEncodeError: + # In case the value is a KimchiException or it formats + # itself to a unicode string. + args[key] = unicode(value)
return unicode(translation.gettext(text), 'utf-8') % args
on 2014/04/03 00:01, Aline Manera wrote:
Regarding the root cause, I don't think we should display an error message with 2 exception
KCHDR0005E: Unable to generate debug report my-debugreport. Details: KCHDR0003E: Unable to create debug report my-debugreport. Details: -9.
It is odd.
We should only raise one exception with all the details.
Agree. I updated it in the v2 patch. I'd also like to keep the decoding improvements for the KimchiException, because it makes KimchiException accepts not only strings and unicode strings, but also objects that formats themselves to strings or unicode strings, which is a common and convenient pattern in exception handling.
On 04/02/2014 02:42 AM, zhshzhou@linux.vnet.ibm.com wrote:
From: Zhou Zheng Sheng <zhshzhou@linux.vnet.ibm.com>
KimchiException provides the ability to translate each argument and the error message. It decodes the translated message to unicode. It's also smart and avoids decoding a unicode argument twice by a check to make sure the to-be-decoded value is not unicode.
The problem is that when a KimchiException is initialized by another KimchiException, for example,
try: ... except OperationFailed as e: ... raise OperationFailed('KCHXXXX', {'err': e})
the variable "e" is not a unicode object so it passes the check mentioned above, then it gets decoded twice and raises an encoding exception.
This patch does not only rely on the type of the to-be-decoded object, but also catches the encoding exeption. It firstly try to call str(obj) to get the ascii string of the obj, then decodes it to unicode. This is valid for a normal string or any object can format itself to a normal string. If it fails, it try to call unicode(obj) to get the unicode string of the object. This is valid for objects can format themselves to a unicode string. In all, it handles various objects like following.
If obj is a unicode string, use it directly. If obj is a normal string, decode it to a unicode string. If obj can format itself to a normal string, format it and decode it to unicode. If obj can format itself to a unicode string, format it.
Another problem is that by the current design, KimchiException always stores error message in unicode. This can cause encoding exception when we call "str(KimchiExceptionObject)". This patch is not a total refactor of the translation and encoding so it does not solve this problem. The workaround is using KimchiExceptionObject.message instead of str it.
Notice: I'm following the Python str.encode and decode semantics in the commit message. "encode" means converting unicode sting to ascii/utf-8. "decode" means converting ascii/utf-8 string to unicode. "encoding" means the action of decode and encode in general.
Signed-off-by: Zhou Zheng Sheng <zhshzhou@linux.vnet.ibm.com> --- src/kimchi/asynctask.py | 2 +- src/kimchi/exception.py | 8 +++++++- 2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/src/kimchi/asynctask.py b/src/kimchi/asynctask.py index 8f0d96c..54fb749 100644 --- a/src/kimchi/asynctask.py +++ b/src/kimchi/asynctask.py @@ -68,4 +68,4 @@ class AsyncTask(object): except Exception, e: cherrypy.log.error_log.error("Error in async_task %s " % self.id) cherrypy.log.error_log.error(traceback.format_exc()) - cb("Unexpected exception: %s" % str(e), False) + cb("Unexpected exception: %s" % e.message, False) diff --git a/src/kimchi/exception.py b/src/kimchi/exception.py index 87982ea..fcf60cc 100644 --- a/src/kimchi/exception.py +++ b/src/kimchi/exception.py @@ -54,7 +54,13 @@ class KimchiException(Exception):
for key, value in args.iteritems(): if not isinstance(value, unicode): - args[key] = unicode(str(value), 'utf-8') + try: + # In case the value formats itself to an ascii string. + args[key] = unicode(str(value), 'utf-8') + except UnicodeEncodeError: + # In case the value is a KimchiException or it formats + # itself to a unicode string. + args[key] = unicode(value)
return unicode(translation.gettext(text), 'utf-8') % args
-- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshzhou@linux.vnet.ibm.com Telephone: 86-10-82454397
participants (4)
-
Aline Manera -
Sheldon -
Zhou Zheng Sheng -
zhshzhou＠linux.vnet.ibm.com