[ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6

Francesco Romani fromani at redhat.com
Mon Nov 10 08:02:44 UTC 2014


Hi everyone,

I was doing JSON-RPC investigation recently, running VDSM on RHEL6.{5,6},
and while some initial profile work, I discovered a (little) performance degradation
about pure JSONRPC coding/encoding, *only* on the above platforms. Here's why:

- the JSON codec module in the python stdlib is based on simplejson
- (thus) simplejson API is identical to stdlib's json module API
- python 2.6 JSON codec is (based on) simplejson 1.9
- python 2.7 JSON codec is (based on) simplejson 2.0.9, with significant speedups. See
  https://docs.python.org/2/whatsnew/2.7.html#new-and-improved-modules and
  https://bugs.python.org/issue4136
- RHEL6.x/Centos 6.x includes python 2.6, so here (and only here) JSON codec
  is unnecessarily slower.
- RHEL6.x (surely) and CentOS 6.x (need to check, don't see why not) includes
  anyway simplejson 2.0.9 as external package

So, it seems to me that we can get python 2.7 speed even on stable platform with
a five line patch:

=== cut here ===
diff --git a/lib/yajsonrpc/__init__.py b/lib/yajsonrpc/__init__.py
index b3fd590..6682fc3 100644
--- a/lib/yajsonrpc/__init__.py
+++ b/lib/yajsonrpc/__init__.py
@@ -12,7 +12,10 @@
 # You should have received a copy of the GNU General Public
 # License along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
-import json
+try:
+    import simplejson as json
+except ImportError
+    import json
 import logging
 from functools import partial
 from Queue import Queue
=== cut here ===

Here's an excerpt of the effects of this patch, using an initial still rough bencmark
- once again 100 VMs boot, is the beginning of every testing routine from mine,
so nothing new here.

Let's consider two initial profiles. Please note that
1) the amount of calls is not the same (error from mine in profiling, need more precise bench)
   but still
2) using 2.0.9 module JSON just vanishes from the profile alltogether

Vanilla VDSM, stdlib json, top 20 expensive calls

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   19/100  127.216    6.696    9.974    0.100 /usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run)
1555/1635   41.180    0.026   58.566    0.036 /usr/lib64/python2.6/threading.py:481(Thread.run)
11708/1553160   11.967    0.001   37.921    0.000 /usr/lib64/python2.6/copy.py:144(deepcopy)
  1268613    6.154    0.000    9.209    0.000 /usr/lib64/python2.6/copy.py:261(_keep_alive)
4890/130093    6.149    0.001   37.596    0.000 /usr/lib64/python2.6/copy.py:251(_deepcopy_dict)
497858/2070078    5.224    0.000   11.102    0.000 /usr/lib64/python2.6/json/encoder.py:284(JSONEncoder._iterencode)
498440/1333474    4.187    0.000   10.368    0.000 /usr/lib64/python2.6/json/encoder.py:213(JSONEncoder._iterencode_dict)
   12/100    4.115    0.343    4.130    0.041 /usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run)
    43826    3.692    0.000    6.516    0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler)
     7345    3.657    0.000   11.047    0.002 /usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats)
9666/320508    3.568    0.000    3.572    0.000 /usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper)
274994/275000    3.339    0.000    5.593    0.000 /usr/lib64/python2.6/StringIO.py:208(StringIO.write)
    90217    1.823    0.000    1.870    0.000 /usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__)
44025/44036    1.666    0.000    2.470    0.000 /usr/share/vdsm/storage/lvm.py:217(makeLV)
144212/144226    1.287    0.000    1.292    0.000 /usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr)
      201    1.285    0.006    7.278    0.036 /usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs)
   109880    1.235    0.000    1.283    0.000 /usr/lib64/python2.6/xml/dom/minidom.py:281(Document._append_child)
    57874    1.163    0.000    1.799    0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata)
  1130859    1.160    0.000    1.160    0.000 /usr/lib64/python2.6/copy.py:197(_deepcopy_atomic)
3021/3022    1.121    0.000    2.554    0.001 /usr/lib64/python2.6/inspect.py:247(getmembers)


patch applied, top 20 expensive calls

   24/100  117.222    4.884   13.372    0.134 /usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run)
1209/1302   60.812    0.050   64.963    0.050 /usr/lib64/python2.6/threading.py:481(Thread.run)
9243/1235834   10.259    0.001   33.047    0.000 /usr/lib64/python2.6/copy.py:144(deepcopy)
   13/100    7.007    0.539    7.281    0.073 /usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run)
4116/103669    5.536    0.001   32.779    0.000 /usr/lib64/python2.6/copy.py:251(_deepcopy_dict)
9670/320560    4.437    0.000    4.443    0.000 /usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper)
     5653    3.232    0.001    9.443    0.002 /usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats)
    43836    3.117    0.000    7.304    0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler)
274967/275000    2.955    0.000    6.833    0.000 /usr/lib64/python2.6/StringIO.py:208(StringIO.write)
1100/28500    2.376    0.002   12.881    0.000 /usr/share/vdsm/virt/vmxml.py:489(Element.__hacked_writexml)
43745/44036    1.729    0.000    2.736    0.000 /usr/share/vdsm/storage/lvm.py:217(makeLV)
90217/90221    1.695    0.000    1.953    0.000 /usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__)
      201    1.549    0.008    7.725    0.038 /usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs)
    57888    1.441    0.000    2.216    0.000 /usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata)
2351/2354    1.262    0.001    2.750    0.001 /usr/lib64/python2.6/inspect.py:247(getmembers)
   992794    1.232    0.000    8.027    0.000 /usr/lib64/python2.6/copy.py:261(_keep_alive)
     4827    1.150    0.000    1.368    0.000 /usr/lib/python2.6/site-packages/mom/Policy/spark.py:211(Parser.buildState)
   108040    1.119    0.000    1.119    0.000 /usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr)
336771/422122    1.106    0.000    2.575    0.000 /usr/share/vdsm/protocoldetector.py:108(MultiProtocolAcceptor._process_events)
   883021    0.987    0.000    0.987    0.000 /usr/lib64/python2.6/copy.py:197(_deepcopy_atomic)


Indeed this is a microbenchmark, but my point is we *recover* some speed at nearly
zero cost and risk.

Now, the question is:
Do we want this patch? Do we want it on master? On 3.5 branch? Only on RHEL/Centos 6 platforms?

-- 
Francesco Romani
RedHat Engineering Virtualization R & D
Phone: 8261328
IRC: fromani



More information about the Devel mailing list