[VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
Hi everyone, I was doing JSON-RPC investigation recently, running VDSM on RHEL6.{5,6}, and while some initial profile work, I discovered a (little) performance degradation about pure JSONRPC coding/encoding, *only* on the above platforms. Here's why: - the JSON codec module in the python stdlib is based on simplejson - (thus) simplejson API is identical to stdlib's json module API - python 2.6 JSON codec is (based on) simplejson 1.9 - python 2.7 JSON codec is (based on) simplejson 2.0.9, with significant speedups. See https://docs.python.org/2/whatsnew/2.7.html#new-and-improved-modules and https://bugs.python.org/issue4136 - RHEL6.x/Centos 6.x includes python 2.6, so here (and only here) JSON codec is unnecessarily slower. - RHEL6.x (surely) and CentOS 6.x (need to check, don't see why not) includes anyway simplejson 2.0.9 as external package So, it seems to me that we can get python 2.7 speed even on stable platform with a five line patch: === cut here === diff --git a/lib/yajsonrpc/__init__.py b/lib/yajsonrpc/__init__.py index b3fd590..6682fc3 100644 --- a/lib/yajsonrpc/__init__.py +++ b/lib/yajsonrpc/__init__.py @@ -12,7 +12,10 @@ # You should have received a copy of the GNU General Public # License along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA -import json +try: + import simplejson as json +except ImportError + import json import logging from functools import partial from Queue import Queue === cut here === Here's an excerpt of the effects of this patch, using an initial still rough bencmark - once again 100 VMs boot, is the beginning of every testing routine from mine, so nothing new here. Let's consider two initial profiles. Please note that 1) the amount of calls is not the same (error from mine in profiling, need more precise bench) but still 2) using 2.0.9 module JSON just vanishes from the profile alltogether Vanilla VDSM, stdlib json, top 20 expensive calls ncalls tottime percall cumtime percall filename:lineno(function) 19/100 127.216 6.696 9.974 0.100 /usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run) 1555/1635 41.180 0.026 58.566 0.036 /usr/lib64/python2.6/threading.py:481(Thread.run) 11708/1553160 11.967 0.001 37.921 0.000 /usr/lib64/python2.6/copy.py:144(deepcopy) 1268613 6.154 0.000 9.209 0.000 /usr/lib64/python2.6/copy.py:261(_keep_alive) 4890/130093 6.149 0.001 37.596 0.000 /usr/lib64/python2.6/copy.py:251(_deepcopy_dict) 497858/2070078 5.224 0.000 11.102 0.000 /usr/lib64/python2.6/json/encoder.py:284(JSONEncoder._iterencode) 498440/1333474 4.187 0.000 10.368 0.000 /usr/lib64/python2.6/json/encoder.py:213(JSONEncoder._iterencode_dict) 12/100 4.115 0.343 4.130 0.041 /usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run) 43826 3.692 0.000 6.516 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler) 7345 3.657 0.000 11.047 0.002 /usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats) 9666/320508 3.568 0.000 3.572 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper) 274994/275000 3.339 0.000 5.593 0.000 /usr/lib64/python2.6/StringIO.py:208(StringIO.write) 90217 1.823 0.000 1.870 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__) 44025/44036 1.666 0.000 2.470 0.000 /usr/share/vdsm/storage/lvm.py:217(makeLV) 144212/144226 1.287 0.000 1.292 0.000 /usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr) 201 1.285 0.006 7.278 0.036 /usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs) 109880 1.235 0.000 1.283 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:281(Document._append_child) 57874 1.163 0.000 1.799 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata) 1130859 1.160 0.000 1.160 0.000 /usr/lib64/python2.6/copy.py:197(_deepcopy_atomic) 3021/3022 1.121 0.000 2.554 0.001 /usr/lib64/python2.6/inspect.py:247(getmembers) patch applied, top 20 expensive calls 24/100 117.222 4.884 13.372 0.134 /usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run) 1209/1302 60.812 0.050 64.963 0.050 /usr/lib64/python2.6/threading.py:481(Thread.run) 9243/1235834 10.259 0.001 33.047 0.000 /usr/lib64/python2.6/copy.py:144(deepcopy) 13/100 7.007 0.539 7.281 0.073 /usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run) 4116/103669 5.536 0.001 32.779 0.000 /usr/lib64/python2.6/copy.py:251(_deepcopy_dict) 9670/320560 4.437 0.000 4.443 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper) 5653 3.232 0.001 9.443 0.002 /usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats) 43836 3.117 0.000 7.304 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler) 274967/275000 2.955 0.000 6.833 0.000 /usr/lib64/python2.6/StringIO.py:208(StringIO.write) 1100/28500 2.376 0.002 12.881 0.000 /usr/share/vdsm/virt/vmxml.py:489(Element.__hacked_writexml) 43745/44036 1.729 0.000 2.736 0.000 /usr/share/vdsm/storage/lvm.py:217(makeLV) 90217/90221 1.695 0.000 1.953 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__) 201 1.549 0.008 7.725 0.038 /usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs) 57888 1.441 0.000 2.216 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata) 2351/2354 1.262 0.001 2.750 0.001 /usr/lib64/python2.6/inspect.py:247(getmembers) 992794 1.232 0.000 8.027 0.000 /usr/lib64/python2.6/copy.py:261(_keep_alive) 4827 1.150 0.000 1.368 0.000 /usr/lib/python2.6/site-packages/mom/Policy/spark.py:211(Parser.buildState) 108040 1.119 0.000 1.119 0.000 /usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr) 336771/422122 1.106 0.000 2.575 0.000 /usr/share/vdsm/protocoldetector.py:108(MultiProtocolAcceptor._process_events) 883021 0.987 0.000 0.987 0.000 /usr/lib64/python2.6/copy.py:197(_deepcopy_atomic) Indeed this is a microbenchmark, but my point is we *recover* some speed at nearly zero cost and risk. Now, the question is: Do we want this patch? Do we want it on master? On 3.5 branch? Only on RHEL/Centos 6 platforms? -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani
Great work, +1 to merge it to all branches. Can you post results of the same test for xmlrpc? I wonder what is the slowdown/speedup between protocols. On Mon, Nov 10, 2014 at 9:02 AM, Francesco Romani <fromani@redhat.com> wrote:
Hi everyone,
I was doing JSON-RPC investigation recently, running VDSM on RHEL6.{5,6}, and while some initial profile work, I discovered a (little) performance degradation about pure JSONRPC coding/encoding, *only* on the above platforms. Here's why:
- the JSON codec module in the python stdlib is based on simplejson - (thus) simplejson API is identical to stdlib's json module API - python 2.6 JSON codec is (based on) simplejson 1.9 - python 2.7 JSON codec is (based on) simplejson 2.0.9, with significant speedups. See https://docs.python.org/2/whatsnew/2.7.html#new-and-improved-modules and https://bugs.python.org/issue4136 - RHEL6.x/Centos 6.x includes python 2.6, so here (and only here) JSON codec is unnecessarily slower. - RHEL6.x (surely) and CentOS 6.x (need to check, don't see why not) includes anyway simplejson 2.0.9 as external package
So, it seems to me that we can get python 2.7 speed even on stable platform with a five line patch:
=== cut here === diff --git a/lib/yajsonrpc/__init__.py b/lib/yajsonrpc/__init__.py index b3fd590..6682fc3 100644 --- a/lib/yajsonrpc/__init__.py +++ b/lib/yajsonrpc/__init__.py @@ -12,7 +12,10 @@ # You should have received a copy of the GNU General Public # License along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA -import json +try: + import simplejson as json +except ImportError + import json import logging from functools import partial from Queue import Queue === cut here ===
Here's an excerpt of the effects of this patch, using an initial still rough bencmark - once again 100 VMs boot, is the beginning of every testing routine from mine, so nothing new here.
Let's consider two initial profiles. Please note that 1) the amount of calls is not the same (error from mine in profiling, need more precise bench) but still 2) using 2.0.9 module JSON just vanishes from the profile alltogether
Vanilla VDSM, stdlib json, top 20 expensive calls
ncalls tottime percall cumtime percall filename:lineno(function) 19/100 127.216 6.696 9.974 0.100 /usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run) 1555/1635 41.180 0.026 58.566 0.036 /usr/lib64/python2.6/threading.py:481(Thread.run) 11708/1553160 11.967 0.001 37.921 0.000 /usr/lib64/python2.6/copy.py:144(deepcopy) 1268613 6.154 0.000 9.209 0.000 /usr/lib64/python2.6/copy.py:261(_keep_alive) 4890/130093 6.149 0.001 37.596 0.000 /usr/lib64/python2.6/copy.py:251(_deepcopy_dict) 497858/2070078 5.224 0.000 11.102 0.000 /usr/lib64/python2.6/json/encoder.py:284(JSONEncoder._iterencode) 498440/1333474 4.187 0.000 10.368 0.000 /usr/lib64/python2.6/json/encoder.py:213(JSONEncoder._iterencode_dict) 12/100 4.115 0.343 4.130 0.041 /usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run) 43826 3.692 0.000 6.516 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler) 7345 3.657 0.000 11.047 0.002 /usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats) 9666/320508 3.568 0.000 3.572 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper) 274994/275000 3.339 0.000 5.593 0.000 /usr/lib64/python2.6/StringIO.py:208(StringIO.write) 90217 1.823 0.000 1.870 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__) 44025/44036 1.666 0.000 2.470 0.000 /usr/share/vdsm/storage/lvm.py:217(makeLV) 144212/144226 1.287 0.000 1.292 0.000 /usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr) 201 1.285 0.006 7.278 0.036 /usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs) 109880 1.235 0.000 1.283 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:281(Document._append_child) 57874 1.163 0.000 1.799 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata) 1130859 1.160 0.000 1.160 0.000 /usr/lib64/python2.6/copy.py:197(_deepcopy_atomic) 3021/3022 1.121 0.000 2.554 0.001 /usr/lib64/python2.6/inspect.py:247(getmembers)
patch applied, top 20 expensive calls
24/100 117.222 4.884 13.372 0.134 /usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run) 1209/1302 60.812 0.050 64.963 0.050 /usr/lib64/python2.6/threading.py:481(Thread.run) 9243/1235834 10.259 0.001 33.047 0.000 /usr/lib64/python2.6/copy.py:144(deepcopy) 13/100 7.007 0.539 7.281 0.073 /usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run) 4116/103669 5.536 0.001 32.779 0.000 /usr/lib64/python2.6/copy.py:251(_deepcopy_dict) 9670/320560 4.437 0.000 4.443 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper) 5653 3.232 0.001 9.443 0.002 /usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats) 43836 3.117 0.000 7.304 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler) 274967/275000 2.955 0.000 6.833 0.000 /usr/lib64/python2.6/StringIO.py:208(StringIO.write) 1100/28500 2.376 0.002 12.881 0.000 /usr/share/vdsm/virt/vmxml.py:489(Element.__hacked_writexml) 43745/44036 1.729 0.000 2.736 0.000 /usr/share/vdsm/storage/lvm.py:217(makeLV) 90217/90221 1.695 0.000 1.953 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__) 201 1.549 0.008 7.725 0.038 /usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs) 57888 1.441 0.000 2.216 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata) 2351/2354 1.262 0.001 2.750 0.001 /usr/lib64/python2.6/inspect.py:247(getmembers) 992794 1.232 0.000 8.027 0.000 /usr/lib64/python2.6/copy.py:261(_keep_alive) 4827 1.150 0.000 1.368 0.000 /usr/lib/python2.6/site-packages/mom/Policy/spark.py:211(Parser.buildState) 108040 1.119 0.000 1.119 0.000 /usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr) 336771/422122 1.106 0.000 2.575 0.000 /usr/share/vdsm/protocoldetector.py:108(MultiProtocolAcceptor._process_events) 883021 0.987 0.000 0.987 0.000 /usr/lib64/python2.6/copy.py:197(_deepcopy_atomic)
Indeed this is a microbenchmark, but my point is we *recover* some speed at nearly zero cost and risk.
Now, the question is: Do we want this patch? Do we want it on master? On 3.5 branch? Only on RHEL/Centos 6 platforms?
-- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
----- Original Message -----
From: "Piotr Kliczewski" <piotr.kliczewski@gmail.com> To: "Francesco Romani" <fromani@redhat.com> Cc: devel@ovirt.org Sent: Monday, November 10, 2014 9:40:22 AM Subject: Re: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
Great work,
+1 to merge it to all branches.
Can you post results of the same test for xmlrpc?
I wonder what is the slowdown/speedup between protocols.
I'm on it, results will follow asap. I just need to figure out a more precise bench, to rule out unneeded noise. Bests, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani
----- Original Message -----
From: "Francesco Romani" <fromani@redhat.com> To: devel@ovirt.org Sent: Monday, November 10, 2014 10:02:44 AM Subject: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
Hi everyone,
I was doing JSON-RPC investigation recently, running VDSM on RHEL6.{5,6}, and while some initial profile work, I discovered a (little) performance degradation about pure JSONRPC coding/encoding, *only* on the above platforms. Here's why:
- the JSON codec module in the python stdlib is based on simplejson - (thus) simplejson API is identical to stdlib's json module API - python 2.6 JSON codec is (based on) simplejson 1.9 - python 2.7 JSON codec is (based on) simplejson 2.0.9, with significant speedups. See https://docs.python.org/2/whatsnew/2.7.html#new-and-improved-modules and https://bugs.python.org/issue4136 - RHEL6.x/Centos 6.x includes python 2.6, so here (and only here) JSON codec is unnecessarily slower. - RHEL6.x (surely) and CentOS 6.x (need to check, don't see why not) includes anyway simplejson 2.0.9 as external package
So, it seems to me that we can get python 2.7 speed even on stable platform with a five line patch:
=== cut here === diff --git a/lib/yajsonrpc/__init__.py b/lib/yajsonrpc/__init__.py index b3fd590..6682fc3 100644 --- a/lib/yajsonrpc/__init__.py +++ b/lib/yajsonrpc/__init__.py @@ -12,7 +12,10 @@ # You should have received a copy of the GNU General Public # License along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA -import json +try: + import simplejson as json +except ImportError + import json import logging from functools import partial from Queue import Queue === cut here ===
We have vdsm/lib/compat.py for this stuff. Then everyone needing json will do "from vdsm.compat import json"
Here's an excerpt of the effects of this patch, using an initial still rough bencmark - once again 100 VMs boot, is the beginning of every testing routine from mine, so nothing new here.
Let's consider two initial profiles. Please note that 1) the amount of calls is not the same (error from mine in profiling, need more precise bench) but still 2) using 2.0.9 module JSON just vanishes from the profile alltogether
Vanilla VDSM, stdlib json, top 20 expensive calls
ncalls tottime percall cumtime percall filename:lineno(function) 19/100 127.216 6.696 9.974 0.100 /usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run) 1555/1635 41.180 0.026 58.566 0.036 /usr/lib64/python2.6/threading.py:481(Thread.run) 11708/1553160 11.967 0.001 37.921 0.000 /usr/lib64/python2.6/copy.py:144(deepcopy) 1268613 6.154 0.000 9.209 0.000 /usr/lib64/python2.6/copy.py:261(_keep_alive) 4890/130093 6.149 0.001 37.596 0.000 /usr/lib64/python2.6/copy.py:251(_deepcopy_dict) 497858/2070078 5.224 0.000 11.102 0.000 /usr/lib64/python2.6/json/encoder.py:284(JSONEncoder._iterencode) 498440/1333474 4.187 0.000 10.368 0.000 /usr/lib64/python2.6/json/encoder.py:213(JSONEncoder._iterencode_dict) 12/100 4.115 0.343 4.130 0.041 /usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run) 43826 3.692 0.000 6.516 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler) 7345 3.657 0.000 11.047 0.002 /usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats) 9666/320508 3.568 0.000 3.572 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper) 274994/275000 3.339 0.000 5.593 0.000 /usr/lib64/python2.6/StringIO.py:208(StringIO.write) 90217 1.823 0.000 1.870 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__) 44025/44036 1.666 0.000 2.470 0.000 /usr/share/vdsm/storage/lvm.py:217(makeLV) 144212/144226 1.287 0.000 1.292 0.000 /usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr) 201 1.285 0.006 7.278 0.036 /usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs) 109880 1.235 0.000 1.283 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:281(Document._append_child) 57874 1.163 0.000 1.799 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata) 1130859 1.160 0.000 1.160 0.000 /usr/lib64/python2.6/copy.py:197(_deepcopy_atomic) 3021/3022 1.121 0.000 2.554 0.001 /usr/lib64/python2.6/inspect.py:247(getmembers)
patch applied, top 20 expensive calls
24/100 117.222 4.884 13.372 0.134 /usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run) 1209/1302 60.812 0.050 64.963 0.050 /usr/lib64/python2.6/threading.py:481(Thread.run) 9243/1235834 10.259 0.001 33.047 0.000 /usr/lib64/python2.6/copy.py:144(deepcopy) 13/100 7.007 0.539 7.281 0.073 /usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run) 4116/103669 5.536 0.001 32.779 0.000 /usr/lib64/python2.6/copy.py:251(_deepcopy_dict) 9670/320560 4.437 0.000 4.443 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper) 5653 3.232 0.001 9.443 0.002 /usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats) 43836 3.117 0.000 7.304 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler) 274967/275000 2.955 0.000 6.833 0.000 /usr/lib64/python2.6/StringIO.py:208(StringIO.write) 1100/28500 2.376 0.002 12.881 0.000 /usr/share/vdsm/virt/vmxml.py:489(Element.__hacked_writexml) 43745/44036 1.729 0.000 2.736 0.000 /usr/share/vdsm/storage/lvm.py:217(makeLV) 90217/90221 1.695 0.000 1.953 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__) 201 1.549 0.008 7.725 0.038 /usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs) 57888 1.441 0.000 2.216 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata) 2351/2354 1.262 0.001 2.750 0.001 /usr/lib64/python2.6/inspect.py:247(getmembers) 992794 1.232 0.000 8.027 0.000 /usr/lib64/python2.6/copy.py:261(_keep_alive) 4827 1.150 0.000 1.368 0.000 /usr/lib/python2.6/site-packages/mom/Policy/spark.py:211(Parser.buildState) 108040 1.119 0.000 1.119 0.000 /usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr) 336771/422122 1.106 0.000 2.575 0.000 /usr/share/vdsm/protocoldetector.py:108(MultiProtocolAcceptor._process_events) 883021 0.987 0.000 0.987 0.000 /usr/lib64/python2.6/copy.py:197(_deepcopy_atomic)
Indeed this is a microbenchmark, but my point is we *recover* some speed at nearly zero cost and risk.
Now, the question is: Do we want this patch? Do we want it on master? On 3.5 branch? Only on RHEL/Centos 6 platforms?
Nice catch - using pure python encoder/decoder is indeed much slower, strange that Python 2.6 does not include the c extension. I think we would like to use simplejson on all platforms, to make the build and support simpler. Nir
----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com> To: "Francesco Romani" <fromani@redhat.com> Cc: devel@ovirt.org Sent: Monday, November 10, 2014 1:21:53 PM Subject: Re: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
We have vdsm/lib/compat.py for this stuff.
Then everyone needing json will do "from vdsm.compat import json"
Do you think is better to have an hard dependency on 'simplejson' or a soft one like I showed above hidden on compat.py? I think we agree that 'explicit is better than implicit' here, so... (see below).
Nice catch - using pure python encoder/decoder is indeed much slower, strange that Python 2.6 does not include the c extension.
I think we would like to use simplejson on all platforms, to make the build and support simpler.
...agreed, and posted http://gerrit.ovirt.org/#/c/35024/ Bests, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani
----- Original Message -----
From: "Francesco Romani" <fromani@redhat.com> To: devel@ovirt.org Cc: "Nir Soffer" <nsoffer@redhat.com> Sent: Tuesday, November 11, 2014 12:24:43 PM Subject: Re: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com> To: "Francesco Romani" <fromani@redhat.com> Cc: devel@ovirt.org Sent: Monday, November 10, 2014 1:21:53 PM Subject: Re: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
We have vdsm/lib/compat.py for this stuff.
Then everyone needing json will do "from vdsm.compat import json"
Do you think is better to have an hard dependency on 'simplejson' or a soft one like I showed above hidden on compat.py?
It depends on the availability of simplejson. If it available on all platforms, we can require it.
I think we agree that 'explicit is better than implicit' here, so... (see below).
I prefer to see "json" in the code, even if we use simplejson library. When we stop supporting Python 2.6 the only change we have to do is remove the "soft" import mechanism. Nir
On Tue, Nov 11, 2014 at 05:31:37AM -0500, Nir Soffer wrote:
----- Original Message -----
From: "Francesco Romani" <fromani@redhat.com> To: devel@ovirt.org Cc: "Nir Soffer" <nsoffer@redhat.com> Sent: Tuesday, November 11, 2014 12:24:43 PM Subject: Re: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com> To: "Francesco Romani" <fromani@redhat.com> Cc: devel@ovirt.org Sent: Monday, November 10, 2014 1:21:53 PM Subject: Re: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
We have vdsm/lib/compat.py for this stuff.
Then everyone needing json will do "from vdsm.compat import json"
Do you think is better to have an hard dependency on 'simplejson' or a soft one like I showed above hidden on compat.py?
It depends on the availability of simplejson. If it available on all platforms, we can require it.
We've learned the hard way that this was not the case.. http://gerrit.ovirt.org/#/c/35354
On 19/11/14 18:36, Dan Kenigsberg wrote:
We've learned the hard way that this was not the case.. http://gerrit.ovirt.org/#/c/35354
Why don't you just require this one package from epel than? epel is included in EL7, you just need to activate it. Furthermore, some engine packages are also just available via epel, so I see no reason to not require epel for vdsm too? -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH & Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
On Thu, Nov 20, 2014 at 11:12:44AM +0100, Sven Kieske wrote:
On 19/11/14 18:36, Dan Kenigsberg wrote:
We've learned the hard way that this was not the case.. http://gerrit.ovirt.org/#/c/35354
Why don't you just require this one package from epel than? epel is included in EL7, you just need to activate it.
Furthermore, some engine packages are also just available via epel, so I see no reason to not require epel for vdsm too?
RHEV cannot depend on EPEL. If a package is not in RHEL, RHEV must consume it, which is something that's not worth doing in this case (where there's not functional benefit). Dan.
On 20/11/14 12:18, Dan Kenigsberg wrote:
RHEV cannot depend on EPEL. If a package is not in RHEL, RHEV must consume it, which is something that's not worth doing in this case (where there's not functional benefit).
As I said: it already does that: an excerpt from the latest ovirt-release35-001-1.noarch.rpm: in /./usr/share/ovirt-release35/ovirt-el6-deps.repo you find: [ovirt-3.5-epel] name=Extra Packages for Enterprise Linux 6 - $basearch #baseurl=http://download.fedoraproject.org/pub/epel/6/$basearch mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=$basearch failovermethod=priority enabled=1 includepkgs=epel-release,python-uinput,puppet,python-lockfile,python-cpopen,python-ordereddict,python-pthreading,python-inotify,python-argparse,novnc,python-ply,python-kitchen,python-daemon,python-websockify,livecd-tools,spice-html5,mom,python-IPy,python-ioprocess,ioprocess,rubygem-rgen gpgcheck=1 gpgkey=https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-6 So why can engine depend on epel and vdsm can not? -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH & Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
On Thu, Nov 20, 2014 at 12:36:20PM +0100, Sven Kieske wrote:
On 20/11/14 12:18, Dan Kenigsberg wrote:
RHEV cannot depend on EPEL. If a package is not in RHEL, RHEV must consume it, which is something that's not worth doing in this case (where there's not functional benefit).
As I said: it already does that:
an excerpt from the latest ovirt-release35-001-1.noarch.rpm: in /./usr/share/ovirt-release35/ovirt-el6-deps.repo you find:
[ovirt-3.5-epel] name=Extra Packages for Enterprise Linux 6 - $basearch #baseurl=http://download.fedoraproject.org/pub/epel/6/$basearch mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=$basearch failovermethod=priority enabled=1 includepkgs=epel-release,python-uinput,puppet,python-lockfile,python-cpopen,python-ordereddict,python-pthreading,python-inotify,python-argparse,novnc,python-ply,python-kitchen,python-daemon,python-websockify,livecd-tools,spice-html5,mom,python-IPy,python-ioprocess,ioprocess,rubygem-rgen gpgcheck=1 gpgkey=https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-6
So why can engine depend on epel and vdsm can not?
engine and vdsm both can, and do, depend on epel. However, RHEV cannot. Taking an epel package has a price of needing to ship it in http://ftp.redhat.com/pub/redhat/rhev-m/3.x/SRPMS/ (see python-inotify there, as a vdsm dependency). In case of simplejson for el7, it simply does not worth it.
On Tue, Nov 11, 2014 at 05:24:43AM -0500, Francesco Romani wrote:
----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com> To: "Francesco Romani" <fromani@redhat.com> Cc: devel@ovirt.org Sent: Monday, November 10, 2014 1:21:53 PM Subject: Re: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
We have vdsm/lib/compat.py for this stuff.
Then everyone needing json will do "from vdsm.compat import json"
Do you think is better to have an hard dependency on 'simplejson' or a soft one like I showed above hidden on compat.py?
I think we agree that 'explicit is better than implicit' here, so... (see below).
I know ganeti has a hard dependency on simplejson for performance reasons, so you wouldn't be the first virtualization project to do so :)
Nice catch - using pure python encoder/decoder is indeed much slower, strange that Python 2.6 does not include the c extension.
I think we would like to use simplejson on all platforms, to make the build and support simpler.
...agreed, and posted http://gerrit.ovirt.org/#/c/35024/
It's well known. The problem is that there is no simplejson in EL6\7 (unless the situation changed since I last checked). I already did extensive profiling on the jsonrpc framework. The patches [1] move us on par with XML-RPC for sequential requests and *way* faster for concurrent commands. There are thoughts about adding msgpack [2] as an optional encoding for responses as it's much smaller and less cpu intensive to encode\decode but we have bigger fish to fry ATM (patches welcome). [1] http://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:sto... [2] http://msgpack.org/ ----- Original Message -----
From: "Francesco Romani" <fromani@redhat.com> To: devel@ovirt.org Sent: Monday, November 10, 2014 10:02:44 AM Subject: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
Hi everyone,
I was doing JSON-RPC investigation recently, running VDSM on RHEL6.{5,6}, and while some initial profile work, I discovered a (little) performance degradation about pure JSONRPC coding/encoding, *only* on the above platforms. Here's why:
- the JSON codec module in the python stdlib is based on simplejson - (thus) simplejson API is identical to stdlib's json module API - python 2.6 JSON codec is (based on) simplejson 1.9 - python 2.7 JSON codec is (based on) simplejson 2.0.9, with significant speedups. See https://docs.python.org/2/whatsnew/2.7.html#new-and-improved-modules and https://bugs.python.org/issue4136 - RHEL6.x/Centos 6.x includes python 2.6, so here (and only here) JSON codec is unnecessarily slower. - RHEL6.x (surely) and CentOS 6.x (need to check, don't see why not) includes anyway simplejson 2.0.9 as external package
So, it seems to me that we can get python 2.7 speed even on stable platform with a five line patch:
=== cut here === diff --git a/lib/yajsonrpc/__init__.py b/lib/yajsonrpc/__init__.py index b3fd590..6682fc3 100644 --- a/lib/yajsonrpc/__init__.py +++ b/lib/yajsonrpc/__init__.py @@ -12,7 +12,10 @@ # You should have received a copy of the GNU General Public # License along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA -import json +try: + import simplejson as json +except ImportError + import json import logging from functools import partial from Queue import Queue === cut here ===
Here's an excerpt of the effects of this patch, using an initial still rough bencmark - once again 100 VMs boot, is the beginning of every testing routine from mine, so nothing new here.
Let's consider two initial profiles. Please note that 1) the amount of calls is not the same (error from mine in profiling, need more precise bench) but still 2) using 2.0.9 module JSON just vanishes from the profile alltogether
Vanilla VDSM, stdlib json, top 20 expensive calls
ncalls tottime percall cumtime percall filename:lineno(function) 19/100 127.216 6.696 9.974 0.100 /usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run) 1555/1635 41.180 0.026 58.566 0.036 /usr/lib64/python2.6/threading.py:481(Thread.run) 11708/1553160 11.967 0.001 37.921 0.000 /usr/lib64/python2.6/copy.py:144(deepcopy) 1268613 6.154 0.000 9.209 0.000 /usr/lib64/python2.6/copy.py:261(_keep_alive) 4890/130093 6.149 0.001 37.596 0.000 /usr/lib64/python2.6/copy.py:251(_deepcopy_dict) 497858/2070078 5.224 0.000 11.102 0.000 /usr/lib64/python2.6/json/encoder.py:284(JSONEncoder._iterencode) 498440/1333474 4.187 0.000 10.368 0.000 /usr/lib64/python2.6/json/encoder.py:213(JSONEncoder._iterencode_dict) 12/100 4.115 0.343 4.130 0.041 /usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run) 43826 3.692 0.000 6.516 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler) 7345 3.657 0.000 11.047 0.002 /usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats) 9666/320508 3.568 0.000 3.572 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper) 274994/275000 3.339 0.000 5.593 0.000 /usr/lib64/python2.6/StringIO.py:208(StringIO.write) 90217 1.823 0.000 1.870 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__) 44025/44036 1.666 0.000 2.470 0.000 /usr/share/vdsm/storage/lvm.py:217(makeLV) 144212/144226 1.287 0.000 1.292 0.000 /usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr) 201 1.285 0.006 7.278 0.036 /usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs) 109880 1.235 0.000 1.283 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:281(Document._append_child) 57874 1.163 0.000 1.799 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata) 1130859 1.160 0.000 1.160 0.000 /usr/lib64/python2.6/copy.py:197(_deepcopy_atomic) 3021/3022 1.121 0.000 2.554 0.001 /usr/lib64/python2.6/inspect.py:247(getmembers)
patch applied, top 20 expensive calls
24/100 117.222 4.884 13.372 0.134 /usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run) 1209/1302 60.812 0.050 64.963 0.050 /usr/lib64/python2.6/threading.py:481(Thread.run) 9243/1235834 10.259 0.001 33.047 0.000 /usr/lib64/python2.6/copy.py:144(deepcopy) 13/100 7.007 0.539 7.281 0.073 /usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run) 4116/103669 5.536 0.001 32.779 0.000 /usr/lib64/python2.6/copy.py:251(_deepcopy_dict) 9670/320560 4.437 0.000 4.443 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper) 5653 3.232 0.001 9.443 0.002 /usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats) 43836 3.117 0.000 7.304 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler) 274967/275000 2.955 0.000 6.833 0.000 /usr/lib64/python2.6/StringIO.py:208(StringIO.write) 1100/28500 2.376 0.002 12.881 0.000 /usr/share/vdsm/virt/vmxml.py:489(Element.__hacked_writexml) 43745/44036 1.729 0.000 2.736 0.000 /usr/share/vdsm/storage/lvm.py:217(makeLV) 90217/90221 1.695 0.000 1.953 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__) 201 1.549 0.008 7.725 0.038 /usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs) 57888 1.441 0.000 2.216 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata) 2351/2354 1.262 0.001 2.750 0.001 /usr/lib64/python2.6/inspect.py:247(getmembers) 992794 1.232 0.000 8.027 0.000 /usr/lib64/python2.6/copy.py:261(_keep_alive) 4827 1.150 0.000 1.368 0.000 /usr/lib/python2.6/site-packages/mom/Policy/spark.py:211(Parser.buildState) 108040 1.119 0.000 1.119 0.000 /usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr) 336771/422122 1.106 0.000 2.575 0.000 /usr/share/vdsm/protocoldetector.py:108(MultiProtocolAcceptor._process_events) 883021 0.987 0.000 0.987 0.000 /usr/lib64/python2.6/copy.py:197(_deepcopy_atomic)
Indeed this is a microbenchmark, but my point is we *recover* some speed at nearly zero cost and risk.
Now, the question is: Do we want this patch? Do we want it on master? On 3.5 branch? Only on RHEL/Centos 6 platforms?
-- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Sun, Nov 23, 2014 at 03:08:59PM -0500, Saggi Mizrahi wrote:
It's well known.
I'd drop the "well" - it was not known to me, and to others.
The problem is that there is no simplejson in EL6\7 (unless the situation changed since I last checked).
Francesco verified that RHEL6 does have simplejson. RHEL7 does not have it (and does not need it as anxiously).
I already did extensive profiling on the jsonrpc framework. The patches [1] move us on par with XML-RPC for sequential requests and *way* faster for concurrent commands.
Thanks!
There are thoughts about adding msgpack [2] as an optional encoding for responses as it's much smaller and less cpu intensive to encode\decode but we have bigger fish to fry ATM (patches welcome).
[1] http://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:sto... [2] http://msgpack.org/
----- Original Message -----
From: "Francesco Romani" <fromani@redhat.com> To: devel@ovirt.org Sent: Monday, November 10, 2014 10:02:44 AM Subject: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
Hi everyone,
I was doing JSON-RPC investigation recently, running VDSM on RHEL6.{5,6}, and while some initial profile work, I discovered a (little) performance degradation about pure JSONRPC coding/encoding, *only* on the above platforms. Here's why:
- the JSON codec module in the python stdlib is based on simplejson - (thus) simplejson API is identical to stdlib's json module API - python 2.6 JSON codec is (based on) simplejson 1.9 - python 2.7 JSON codec is (based on) simplejson 2.0.9, with significant speedups. See https://docs.python.org/2/whatsnew/2.7.html#new-and-improved-modules and https://bugs.python.org/issue4136 - RHEL6.x/Centos 6.x includes python 2.6, so here (and only here) JSON codec is unnecessarily slower. - RHEL6.x (surely) and CentOS 6.x (need to check, don't see why not) includes anyway simplejson 2.0.9 as external package
So, it seems to me that we can get python 2.7 speed even on stable platform with a five line patch:
=== cut here === diff --git a/lib/yajsonrpc/__init__.py b/lib/yajsonrpc/__init__.py index b3fd590..6682fc3 100644 --- a/lib/yajsonrpc/__init__.py +++ b/lib/yajsonrpc/__init__.py @@ -12,7 +12,10 @@ # You should have received a copy of the GNU General Public # License along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA -import json +try: + import simplejson as json +except ImportError + import json import logging from functools import partial from Queue import Queue === cut here ===
Here's an excerpt of the effects of this patch, using an initial still rough bencmark - once again 100 VMs boot, is the beginning of every testing routine from mine, so nothing new here.
Let's consider two initial profiles. Please note that 1) the amount of calls is not the same (error from mine in profiling, need more precise bench) but still 2) using 2.0.9 module JSON just vanishes from the profile alltogether
Vanilla VDSM, stdlib json, top 20 expensive calls
ncalls tottime percall cumtime percall filename:lineno(function) 19/100 127.216 6.696 9.974 0.100 /usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run) 1555/1635 41.180 0.026 58.566 0.036 /usr/lib64/python2.6/threading.py:481(Thread.run) 11708/1553160 11.967 0.001 37.921 0.000 /usr/lib64/python2.6/copy.py:144(deepcopy) 1268613 6.154 0.000 9.209 0.000 /usr/lib64/python2.6/copy.py:261(_keep_alive) 4890/130093 6.149 0.001 37.596 0.000 /usr/lib64/python2.6/copy.py:251(_deepcopy_dict) 497858/2070078 5.224 0.000 11.102 0.000 /usr/lib64/python2.6/json/encoder.py:284(JSONEncoder._iterencode) 498440/1333474 4.187 0.000 10.368 0.000 /usr/lib64/python2.6/json/encoder.py:213(JSONEncoder._iterencode_dict) 12/100 4.115 0.343 4.130 0.041 /usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run) 43826 3.692 0.000 6.516 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler) 7345 3.657 0.000 11.047 0.002 /usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats) 9666/320508 3.568 0.000 3.572 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper) 274994/275000 3.339 0.000 5.593 0.000 /usr/lib64/python2.6/StringIO.py:208(StringIO.write) 90217 1.823 0.000 1.870 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__) 44025/44036 1.666 0.000 2.470 0.000 /usr/share/vdsm/storage/lvm.py:217(makeLV) 144212/144226 1.287 0.000 1.292 0.000 /usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr) 201 1.285 0.006 7.278 0.036 /usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs) 109880 1.235 0.000 1.283 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:281(Document._append_child) 57874 1.163 0.000 1.799 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata) 1130859 1.160 0.000 1.160 0.000 /usr/lib64/python2.6/copy.py:197(_deepcopy_atomic) 3021/3022 1.121 0.000 2.554 0.001 /usr/lib64/python2.6/inspect.py:247(getmembers)
patch applied, top 20 expensive calls
24/100 117.222 4.884 13.372 0.134 /usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run) 1209/1302 60.812 0.050 64.963 0.050 /usr/lib64/python2.6/threading.py:481(Thread.run) 9243/1235834 10.259 0.001 33.047 0.000 /usr/lib64/python2.6/copy.py:144(deepcopy) 13/100 7.007 0.539 7.281 0.073 /usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run) 4116/103669 5.536 0.001 32.779 0.000 /usr/lib64/python2.6/copy.py:251(_deepcopy_dict) 9670/320560 4.437 0.000 4.443 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper) 5653 3.232 0.001 9.443 0.002 /usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats) 43836 3.117 0.000 7.304 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler) 274967/275000 2.955 0.000 6.833 0.000 /usr/lib64/python2.6/StringIO.py:208(StringIO.write) 1100/28500 2.376 0.002 12.881 0.000 /usr/share/vdsm/virt/vmxml.py:489(Element.__hacked_writexml) 43745/44036 1.729 0.000 2.736 0.000 /usr/share/vdsm/storage/lvm.py:217(makeLV) 90217/90221 1.695 0.000 1.953 0.000 /usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__) 201 1.549 0.008 7.725 0.038 /usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs) 57888 1.441 0.000 2.216 0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata) 2351/2354 1.262 0.001 2.750 0.001 /usr/lib64/python2.6/inspect.py:247(getmembers) 992794 1.232 0.000 8.027 0.000 /usr/lib64/python2.6/copy.py:261(_keep_alive) 4827 1.150 0.000 1.368 0.000 /usr/lib/python2.6/site-packages/mom/Policy/spark.py:211(Parser.buildState) 108040 1.119 0.000 1.119 0.000 /usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr) 336771/422122 1.106 0.000 2.575 0.000 /usr/share/vdsm/protocoldetector.py:108(MultiProtocolAcceptor._process_events) 883021 0.987 0.000 0.987 0.000 /usr/lib64/python2.6/copy.py:197(_deepcopy_atomic)
Indeed this is a microbenchmark, but my point is we *recover* some speed at nearly zero cost and risk.
Now, the question is: Do we want this patch? Do we want it on master? On 3.5 branch? Only on RHEL/Centos 6 platforms?
-- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
participants (7)
-
Dan Kenigsberg -
Ewoud Kohl van Wijngaarden -
Francesco Romani -
Nir Soffer -
Piotr Kliczewski -
Saggi Mizrahi -
Sven Kieske