2:02 a.m.
Hi everyone,
I was doing JSON-RPC investigation recently, running VDSM on RHEL6.{5,6},
and while some initial profile work, I discovered a (little) performance degradation
about pure JSONRPC coding/encoding, *only* on the above platforms. Here's why:
- the JSON codec module in the python stdlib is based on simplejson
- (thus) simplejson API is identical to stdlib's json module API
- python 2.6 JSON codec is (based on) simplejson 1.9
- python 2.7 JSON codec is (based on) simplejson 2.0.9, with significant speedups. See
https://docs.python.org/2/whatsnew/2.7.html#new-and-improved-modules and
https://bugs.python.org/issue4136
- RHEL6.x/Centos 6.x includes python 2.6, so here (and only here) JSON codec
is unnecessarily slower.
- RHEL6.x (surely) and CentOS 6.x (need to check, don't see why not) includes
anyway simplejson 2.0.9 as external package
So, it seems to me that we can get python 2.7 speed even on stable platform with
a five line patch:
=== cut here ===
diff --git a/lib/yajsonrpc/__init__.py b/lib/yajsonrpc/__init__.py
index b3fd590..6682fc3 100644
--- a/lib/yajsonrpc/__init__.py
+++ b/lib/yajsonrpc/__init__.py
@@ -12,7 +12,10 @@
# You should have received a copy of the GNU General Public
# License along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
-import json
+try:
+ import simplejson as json
+except ImportError
+ import json
import logging
from functools import partial
from Queue import Queue
=== cut here ===
Here's an excerpt of the effects of this patch, using an initial still rough bencmark
- once again 100 VMs boot, is the beginning of every testing routine from mine,
so nothing new here.
Let's consider two initial profiles. Please note that
1) the amount of calls is not the same (error from mine in profiling, need more precise
bench)
but still
2) using 2.0.9 module JSON just vanishes from the profile alltogether
Vanilla VDSM, stdlib json, top 20 expensive calls
ncalls tottime percall cumtime percall filename:lineno(function)
19/100 127.216 6.696 9.974 0.100
/usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run)
1555/1635 41.180 0.026 58.566 0.036
/usr/lib64/python2.6/threading.py:481(Thread.run)
11708/1553160 11.967 0.001 37.921 0.000
/usr/lib64/python2.6/copy.py:144(deepcopy)
1268613 6.154 0.000 9.209 0.000
/usr/lib64/python2.6/copy.py:261(_keep_alive)
4890/130093 6.149 0.001 37.596 0.000
/usr/lib64/python2.6/copy.py:251(_deepcopy_dict)
497858/2070078 5.224 0.000 11.102 0.000
/usr/lib64/python2.6/json/encoder.py:284(JSONEncoder._iterencode)
498440/1333474 4.187 0.000 10.368 0.000
/usr/lib64/python2.6/json/encoder.py:213(JSONEncoder._iterencode_dict)
12/100 4.115 0.343 4.130 0.041
/usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run)
43826 3.692 0.000 6.516 0.000
/usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler)
7345 3.657 0.000 11.047 0.002
/usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats)
9666/320508 3.568 0.000 3.572 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper)
274994/275000 3.339 0.000 5.593 0.000
/usr/lib64/python2.6/StringIO.py:208(StringIO.write)
90217 1.823 0.000 1.870 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__)
44025/44036 1.666 0.000 2.470 0.000
/usr/share/vdsm/storage/lvm.py:217(makeLV)
144212/144226 1.287 0.000 1.292 0.000
/usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr)
201 1.285 0.006 7.278 0.036
/usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs)
109880 1.235 0.000 1.283 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:281(Document._append_child)
57874 1.163 0.000 1.799 0.000
/usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata)
1130859 1.160 0.000 1.160 0.000
/usr/lib64/python2.6/copy.py:197(_deepcopy_atomic)
3021/3022 1.121 0.000 2.554 0.001
/usr/lib64/python2.6/inspect.py:247(getmembers)
patch applied, top 20 expensive calls
24/100 117.222 4.884 13.372 0.134
/usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run)
1209/1302 60.812 0.050 64.963 0.050
/usr/lib64/python2.6/threading.py:481(Thread.run)
9243/1235834 10.259 0.001 33.047 0.000
/usr/lib64/python2.6/copy.py:144(deepcopy)
13/100 7.007 0.539 7.281 0.073
/usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run)
4116/103669 5.536 0.001 32.779 0.000
/usr/lib64/python2.6/copy.py:251(_deepcopy_dict)
9670/320560 4.437 0.000 4.443 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper)
5653 3.232 0.001 9.443 0.002
/usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats)
43836 3.117 0.000 7.304 0.000
/usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler)
274967/275000 2.955 0.000 6.833 0.000
/usr/lib64/python2.6/StringIO.py:208(StringIO.write)
1100/28500 2.376 0.002 12.881 0.000
/usr/share/vdsm/virt/vmxml.py:489(Element.__hacked_writexml)
43745/44036 1.729 0.000 2.736 0.000
/usr/share/vdsm/storage/lvm.py:217(makeLV)
90217/90221 1.695 0.000 1.953 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__)
201 1.549 0.008 7.725 0.038
/usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs)
57888 1.441 0.000 2.216 0.000
/usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata)
2351/2354 1.262 0.001 2.750 0.001
/usr/lib64/python2.6/inspect.py:247(getmembers)
992794 1.232 0.000 8.027 0.000
/usr/lib64/python2.6/copy.py:261(_keep_alive)
4827 1.150 0.000 1.368 0.000
/usr/lib/python2.6/site-packages/mom/Policy/spark.py:211(Parser.buildState)
108040 1.119 0.000 1.119 0.000
/usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr)
336771/422122 1.106 0.000 2.575 0.000
/usr/share/vdsm/protocoldetector.py:108(MultiProtocolAcceptor._process_events)
883021 0.987 0.000 0.987 0.000
/usr/lib64/python2.6/copy.py:197(_deepcopy_atomic)
Indeed this is a microbenchmark, but my point is we *recover* some speed at nearly
zero cost and risk.
Now, the question is:
Do we want this patch? Do we want it on master? On 3.5 branch? Only on RHEL/Centos 6
platforms?
--
Francesco Romani
RedHat Engineering Virtualization R & D
Phone: 8261328
IRC: fromani
2:40 a.m.
Great work,
+1 to merge it to all branches.
Can you post results of the same test for xmlrpc?
I wonder what is the slowdown/speedup between protocols.
On Mon, Nov 10, 2014 at 9:02 AM, Francesco Romani <fromani(a)redhat.com> wrote:
Hi everyone,
I was doing JSON-RPC investigation recently, running VDSM on RHEL6.{5,6},
and while some initial profile work, I discovered a (little) performance degradation
about pure JSONRPC coding/encoding, *only* on the above platforms. Here's why:
- the JSON codec module in the python stdlib is based on simplejson
- (thus) simplejson API is identical to stdlib's json module API
- python 2.6 JSON codec is (based on) simplejson 1.9
- python 2.7 JSON codec is (based on) simplejson 2.0.9, with significant speedups. See
https://docs.python.org/2/whatsnew/2.7.html#new-and-improved-modules and
https://bugs.python.org/issue4136
- RHEL6.x/Centos 6.x includes python 2.6, so here (and only here) JSON codec
is unnecessarily slower.
- RHEL6.x (surely) and CentOS 6.x (need to check, don't see why not) includes
anyway simplejson 2.0.9 as external package
So, it seems to me that we can get python 2.7 speed even on stable platform with
a five line patch:
=== cut here ===
diff --git a/lib/yajsonrpc/__init__.py b/lib/yajsonrpc/__init__.py
index b3fd590..6682fc3 100644
--- a/lib/yajsonrpc/__init__.py
+++ b/lib/yajsonrpc/__init__.py
@@ -12,7 +12,10 @@
# You should have received a copy of the GNU General Public
# License along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
-import json
+try:
+ import simplejson as json
+except ImportError
+ import json
import logging
from functools import partial
from Queue import Queue
=== cut here ===
Here's an excerpt of the effects of this patch, using an initial still rough
bencmark
- once again 100 VMs boot, is the beginning of every testing routine from mine,
so nothing new here.
Let's consider two initial profiles. Please note that
1) the amount of calls is not the same (error from mine in profiling, need more precise
bench)
but still
2) using 2.0.9 module JSON just vanishes from the profile alltogether
Vanilla VDSM, stdlib json, top 20 expensive calls
ncalls tottime percall cumtime percall filename:lineno(function)
19/100 127.216 6.696 9.974 0.100
/usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run)
1555/1635 41.180 0.026 58.566 0.036
/usr/lib64/python2.6/threading.py:481(Thread.run)
11708/1553160 11.967 0.001 37.921 0.000
/usr/lib64/python2.6/copy.py:144(deepcopy)
1268613 6.154 0.000 9.209 0.000
/usr/lib64/python2.6/copy.py:261(_keep_alive)
4890/130093 6.149 0.001 37.596 0.000
/usr/lib64/python2.6/copy.py:251(_deepcopy_dict)
497858/2070078 5.224 0.000 11.102 0.000
/usr/lib64/python2.6/json/encoder.py:284(JSONEncoder._iterencode)
498440/1333474 4.187 0.000 10.368 0.000
/usr/lib64/python2.6/json/encoder.py:213(JSONEncoder._iterencode_dict)
12/100 4.115 0.343 4.130 0.041
/usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run)
43826 3.692 0.000 6.516 0.000
/usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler)
7345 3.657 0.000 11.047 0.002
/usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats)
9666/320508 3.568 0.000 3.572 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper)
274994/275000 3.339 0.000 5.593 0.000
/usr/lib64/python2.6/StringIO.py:208(StringIO.write)
90217 1.823 0.000 1.870 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__)
44025/44036 1.666 0.000 2.470 0.000
/usr/share/vdsm/storage/lvm.py:217(makeLV)
144212/144226 1.287 0.000 1.292 0.000
/usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr)
201 1.285 0.006 7.278 0.036
/usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs)
109880 1.235 0.000 1.283 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:281(Document._append_child)
57874 1.163 0.000 1.799 0.000
/usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata)
1130859 1.160 0.000 1.160 0.000
/usr/lib64/python2.6/copy.py:197(_deepcopy_atomic)
3021/3022 1.121 0.000 2.554 0.001
/usr/lib64/python2.6/inspect.py:247(getmembers)
patch applied, top 20 expensive calls
24/100 117.222 4.884 13.372 0.134
/usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run)
1209/1302 60.812 0.050 64.963 0.050
/usr/lib64/python2.6/threading.py:481(Thread.run)
9243/1235834 10.259 0.001 33.047 0.000
/usr/lib64/python2.6/copy.py:144(deepcopy)
13/100 7.007 0.539 7.281 0.073
/usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run)
4116/103669 5.536 0.001 32.779 0.000
/usr/lib64/python2.6/copy.py:251(_deepcopy_dict)
9670/320560 4.437 0.000 4.443 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper)
5653 3.232 0.001 9.443 0.002
/usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats)
43836 3.117 0.000 7.304 0.000
/usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler)
274967/275000 2.955 0.000 6.833 0.000
/usr/lib64/python2.6/StringIO.py:208(StringIO.write)
1100/28500 2.376 0.002 12.881 0.000
/usr/share/vdsm/virt/vmxml.py:489(Element.__hacked_writexml)
43745/44036 1.729 0.000 2.736 0.000
/usr/share/vdsm/storage/lvm.py:217(makeLV)
90217/90221 1.695 0.000 1.953 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__)
201 1.549 0.008 7.725 0.038
/usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs)
57888 1.441 0.000 2.216 0.000
/usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata)
2351/2354 1.262 0.001 2.750 0.001
/usr/lib64/python2.6/inspect.py:247(getmembers)
992794 1.232 0.000 8.027 0.000
/usr/lib64/python2.6/copy.py:261(_keep_alive)
4827 1.150 0.000 1.368 0.000
/usr/lib/python2.6/site-packages/mom/Policy/spark.py:211(Parser.buildState)
108040 1.119 0.000 1.119 0.000
/usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr)
336771/422122 1.106 0.000 2.575 0.000
/usr/share/vdsm/protocoldetector.py:108(MultiProtocolAcceptor._process_events)
883021 0.987 0.000 0.987 0.000
/usr/lib64/python2.6/copy.py:197(_deepcopy_atomic)
Indeed this is a microbenchmark, but my point is we *recover* some speed at nearly
zero cost and risk.
Now, the question is:
Do we want this patch? Do we want it on master? On 3.5 branch? Only on RHEL/Centos 6
platforms?
--
Francesco Romani
RedHat Engineering Virtualization R & D
Phone: 8261328
IRC: fromani
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
2:44 a.m.
----- Original Message -----
From: "Piotr Kliczewski"
<piotr.kliczewski(a)gmail.com>
To: "Francesco Romani" <fromani(a)redhat.com>
Cc: devel(a)ovirt.org
Sent: Monday, November 10, 2014 9:40:22 AM
Subject: Re: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance on
RHEL/Centos 6
Great work,
+1 to merge it to all branches.
Can you post results of the same test for xmlrpc?
I wonder what is the slowdown/speedup between protocols.
I'm on it, results will follow asap.
I just need to figure out a more precise bench, to rule out unneeded noise.
Bests,
--
Francesco Romani
RedHat Engineering Virtualization R & D
Phone: 8261328
IRC: fromani
6:21 a.m.
New subject: [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
----- Original Message -----
From: "Francesco Romani" <fromani(a)redhat.com>
To: devel(a)ovirt.org
Sent: Monday, November 10, 2014 10:02:44 AM
Subject: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
Hi everyone,
I was doing JSON-RPC investigation recently, running VDSM on RHEL6.{5,6},
and while some initial profile work, I discovered a (little) performance
degradation
about pure JSONRPC coding/encoding, *only* on the above platforms. Here's
why:
- the JSON codec module in the python stdlib is based on simplejson
- (thus) simplejson API is identical to stdlib's json module API
- python 2.6 JSON codec is (based on) simplejson 1.9
- python 2.7 JSON codec is (based on) simplejson 2.0.9, with significant
speedups. See
https://docs.python.org/2/whatsnew/2.7.html#new-and-improved-modules and
https://bugs.python.org/issue4136
- RHEL6.x/Centos 6.x includes python 2.6, so here (and only here) JSON codec
is unnecessarily slower.
- RHEL6.x (surely) and CentOS 6.x (need to check, don't see why not) includes
anyway simplejson 2.0.9 as external package
So, it seems to me that we can get python 2.7 speed even on stable platform
with
a five line patch:
=== cut here ===
diff --git a/lib/yajsonrpc/__init__.py b/lib/yajsonrpc/__init__.py
index b3fd590..6682fc3 100644
--- a/lib/yajsonrpc/__init__.py
+++ b/lib/yajsonrpc/__init__.py
@@ -12,7 +12,10 @@
# You should have received a copy of the GNU General Public
# License along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
-import json
+try:
+ import simplejson as json
+except ImportError
+ import json
import logging
from functools import partial
from Queue import Queue
=== cut here ===
We have vdsm/lib/compat.py for this stuff.
Then everyone needing json will do "from vdsm.compat import json"
Here's an excerpt of the effects of this patch, using an initial still rough
bencmark
- once again 100 VMs boot, is the beginning of every testing routine from
mine,
so nothing new here.
Let's consider two initial profiles. Please note that
1) the amount of calls is not the same (error from mine in profiling, need
more precise bench)
but still
2) using 2.0.9 module JSON just vanishes from the profile alltogether
Vanilla VDSM, stdlib json, top 20 expensive calls
ncalls tottime percall cumtime percall filename:lineno(function)
19/100 127.216 6.696 9.974 0.100
/usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run)
1555/1635 41.180 0.026 58.566 0.036
/usr/lib64/python2.6/threading.py:481(Thread.run)
11708/1553160 11.967 0.001 37.921 0.000
/usr/lib64/python2.6/copy.py:144(deepcopy)
1268613 6.154 0.000 9.209 0.000
/usr/lib64/python2.6/copy.py:261(_keep_alive)
4890/130093 6.149 0.001 37.596 0.000
/usr/lib64/python2.6/copy.py:251(_deepcopy_dict)
497858/2070078 5.224 0.000 11.102 0.000
/usr/lib64/python2.6/json/encoder.py:284(JSONEncoder._iterencode)
498440/1333474 4.187 0.000 10.368 0.000
/usr/lib64/python2.6/json/encoder.py:213(JSONEncoder._iterencode_dict)
12/100 4.115 0.343 4.130 0.041
/usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run)
43826 3.692 0.000 6.516 0.000
/usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler)
7345 3.657 0.000 11.047 0.002
/usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats)
9666/320508 3.568 0.000 3.572 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper)
274994/275000 3.339 0.000 5.593 0.000
/usr/lib64/python2.6/StringIO.py:208(StringIO.write)
90217 1.823 0.000 1.870 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__)
44025/44036 1.666 0.000 2.470 0.000
/usr/share/vdsm/storage/lvm.py:217(makeLV)
144212/144226 1.287 0.000 1.292 0.000
/usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr)
201 1.285 0.006 7.278 0.036
/usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs)
109880 1.235 0.000 1.283 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:281(Document._append_child)
57874 1.163 0.000 1.799 0.000
/usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata)
1130859 1.160 0.000 1.160 0.000
/usr/lib64/python2.6/copy.py:197(_deepcopy_atomic)
3021/3022 1.121 0.000 2.554 0.001
/usr/lib64/python2.6/inspect.py:247(getmembers)
patch applied, top 20 expensive calls
24/100 117.222 4.884 13.372 0.134
/usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run)
1209/1302 60.812 0.050 64.963 0.050
/usr/lib64/python2.6/threading.py:481(Thread.run)
9243/1235834 10.259 0.001 33.047 0.000
/usr/lib64/python2.6/copy.py:144(deepcopy)
13/100 7.007 0.539 7.281 0.073
/usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run)
4116/103669 5.536 0.001 32.779 0.000
/usr/lib64/python2.6/copy.py:251(_deepcopy_dict)
9670/320560 4.437 0.000 4.443 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper)
5653 3.232 0.001 9.443 0.002
/usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats)
43836 3.117 0.000 7.304 0.000
/usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler)
274967/275000 2.955 0.000 6.833 0.000
/usr/lib64/python2.6/StringIO.py:208(StringIO.write)
1100/28500 2.376 0.002 12.881 0.000
/usr/share/vdsm/virt/vmxml.py:489(Element.__hacked_writexml)
43745/44036 1.729 0.000 2.736 0.000
/usr/share/vdsm/storage/lvm.py:217(makeLV)
90217/90221 1.695 0.000 1.953 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__)
201 1.549 0.008 7.725 0.038
/usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs)
57888 1.441 0.000 2.216 0.000
/usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata)
2351/2354 1.262 0.001 2.750 0.001
/usr/lib64/python2.6/inspect.py:247(getmembers)
992794 1.232 0.000 8.027 0.000
/usr/lib64/python2.6/copy.py:261(_keep_alive)
4827 1.150 0.000 1.368 0.000
/usr/lib/python2.6/site-packages/mom/Policy/spark.py:211(Parser.buildState)
108040 1.119 0.000 1.119 0.000
/usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr)
336771/422122 1.106 0.000 2.575 0.000
/usr/share/vdsm/protocoldetector.py:108(MultiProtocolAcceptor._process_events)
883021 0.987 0.000 0.987 0.000
/usr/lib64/python2.6/copy.py:197(_deepcopy_atomic)
Indeed this is a microbenchmark, but my point is we *recover* some speed at
nearly
zero cost and risk.
Now, the question is:
Do we want this patch? Do we want it on master? On 3.5 branch? Only on
RHEL/Centos 6 platforms?
Nice catch - using pure python encoder/decoder is indeed much slower, strange
that Python 2.6 does not include the c extension.
I think we would like to use simplejson on all platforms, to make the build
and support simpler.
Nir
4:24 a.m.
New subject: [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
----- Original Message -----
From: "Nir Soffer" <nsoffer(a)redhat.com>
To: "Francesco Romani" <fromani(a)redhat.com>
Cc: devel(a)ovirt.org
Sent: Monday, November 10, 2014 1:21:53 PM
Subject: Re: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance
on RHEL/Centos 6
We have vdsm/lib/compat.py for this stuff.
Then everyone needing json will do "from vdsm.compat import json"
Do you think is better to have an hard dependency on 'simplejson'
or a soft one like I showed above hidden on compat.py?
I think we agree that 'explicit is better than implicit' here, so... (see below).
Nice catch - using pure python encoder/decoder is indeed much slower,
strange
that Python 2.6 does not include the c extension.
I think we would like to use simplejson on all platforms, to make the build
and support simpler.
...agreed, and posted http://gerrit.ovirt.org/#/c/35024/
Bests,
--
Francesco Romani
RedHat Engineering Virtualization R & D
Phone: 8261328
IRC: fromani
4:31 a.m.
New subject: [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
----- Original Message -----
From: "Francesco Romani" <fromani(a)redhat.com>
To: devel(a)ovirt.org
Cc: "Nir Soffer" <nsoffer(a)redhat.com>
Sent: Tuesday, November 11, 2014 12:24:43 PM
Subject: Re: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance
on RHEL/Centos 6
----- Original Message -----
> From: "Nir Soffer" <nsoffer(a)redhat.com>
> To: "Francesco Romani" <fromani(a)redhat.com>
> Cc: devel(a)ovirt.org
> Sent: Monday, November 10, 2014 1:21:53 PM
> Subject: Re: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance
> on RHEL/Centos 6
>
> We have vdsm/lib/compat.py for this stuff.
>
> Then everyone needing json will do "from vdsm.compat import json"
Do you think is better to have an hard dependency on 'simplejson'
or a soft one like I showed above hidden on compat.py?
It depends on the availability of simplejson. If it available
on all platforms, we can require it.
I think we agree that 'explicit is better than implicit' here, so... (see
below).
I prefer to see "json" in the code, even if we use simplejson library.
When we stop supporting Python 2.6 the only change we have to do is remove the
"soft" import mechanism.
Nir
11:36 a.m.
New subject: [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
On Tue, Nov 11, 2014 at 05:31:37AM -0500, Nir Soffer wrote:
----- Original Message -----
> From: "Francesco Romani" <fromani(a)redhat.com>
> To: devel(a)ovirt.org
> Cc: "Nir Soffer" <nsoffer(a)redhat.com>
> Sent: Tuesday, November 11, 2014 12:24:43 PM
> Subject: Re: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance
on RHEL/Centos 6
>
>
>
> ----- Original Message -----
> > From: "Nir Soffer" <nsoffer(a)redhat.com>
> > To: "Francesco Romani" <fromani(a)redhat.com>
> > Cc: devel(a)ovirt.org
> > Sent: Monday, November 10, 2014 1:21:53 PM
> > Subject: Re: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance
> > on RHEL/Centos 6
> >
> > We have vdsm/lib/compat.py for this stuff.
> >
> > Then everyone needing json will do "from vdsm.compat import json"
>
> Do you think is better to have an hard dependency on 'simplejson'
> or a soft one like I showed above hidden on compat.py?
It depends on the availability of simplejson. If it available
on all platforms, we can require it.
We've learned the hard way that this was not the case..
http://gerrit.ovirt.org/#/c/35354
4:12 a.m.
New subject: [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
On 19/11/14 18:36, Dan Kenigsberg wrote:
We've learned the hard way that this was not the case..
http://gerrit.ovirt.org/#/c/35354
Why don't you just require this one package from epel than?
epel is included in EL7, you just need to activate it.
Furthermore, some engine packages are also just available
via epel, so I see no reason to not require epel for vdsm too?
--
Mit freundlichen Grüßen / Regards
Sven Kieske
Systemadministrator
Mittwald CM Service GmbH & Co. KG
Königsberger Straße 6
32339 Espelkamp
T: +49-5772-293-100
F: +49-5772-293-333
https://www.mittwald.de
Geschäftsführer: Robert Meyer
St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen
Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
5:18 a.m.
New subject: [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
On Thu, Nov 20, 2014 at 11:12:44AM +0100, Sven Kieske wrote:
On 19/11/14 18:36, Dan Kenigsberg wrote:
> We've learned the hard way that this was not the case..
> http://gerrit.ovirt.org/#/c/35354
Why don't you just require this one package from epel than?
epel is included in EL7, you just need to activate it.
Furthermore, some engine packages are also just available
via epel, so I see no reason to not require epel for vdsm too?
RHEV cannot depend on EPEL. If a package is not in RHEL, RHEV must
consume it, which is something that's not worth doing in this case
(where there's not functional benefit).
Dan.
5:36 a.m.
New subject: [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
On 20/11/14 12:18, Dan Kenigsberg wrote:
RHEV cannot depend on EPEL. If a package is not in RHEL, RHEV must
consume it, which is something that's not worth doing in this case
(where there's not functional benefit).
As I said: it already does that:
an excerpt from the latest ovirt-release35-001-1.noarch.rpm:
in /./usr/share/ovirt-release35/ovirt-el6-deps.repo
you find:
[ovirt-3.5-epel]
name=Extra Packages for Enterprise Linux 6 - $basearch
#baseurl=http://download.fedoraproject.org/pub/epel/6/$basearch
mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-6&arc...
failovermethod=priority
enabled=1
includepkgs=epel-release,python-uinput,puppet,python-lockfile,python-cpopen,python-ordereddict,python-pthreading,python-inotify,python-argparse,novnc,python-ply,python-kitchen,python-daemon,python-websockify,livecd-tools,spice-html5,mom,python-IPy,python-ioprocess,ioprocess,rubygem-rgen
gpgcheck=1
gpgkey=https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-6
So why can engine depend on epel and vdsm can not?
--
Mit freundlichen Grüßen / Regards
Sven Kieske
Systemadministrator
Mittwald CM Service GmbH & Co. KG
Königsberger Straße 6
32339 Espelkamp
T: +49-5772-293-100
F: +49-5772-293-333
https://www.mittwald.de
Geschäftsführer: Robert Meyer
St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen
Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
6:27 a.m.
New subject: [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
On Thu, Nov 20, 2014 at 12:36:20PM +0100, Sven Kieske wrote:
On 20/11/14 12:18, Dan Kenigsberg wrote:
> RHEV cannot depend on EPEL. If a package is not in RHEL, RHEV must
> consume it, which is something that's not worth doing in this case
> (where there's not functional benefit).
As I said: it already does that:
an excerpt from the latest ovirt-release35-001-1.noarch.rpm:
in /./usr/share/ovirt-release35/ovirt-el6-deps.repo
you find:
[ovirt-3.5-epel]
name=Extra Packages for Enterprise Linux 6 - $basearch
#baseurl=http://download.fedoraproject.org/pub/epel/6/$basearch
mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-6&arc...
failovermethod=priority
enabled=1
includepkgs=epel-release,python-uinput,puppet,python-lockfile,python-cpopen,python-ordereddict,python-pthreading,python-inotify,python-argparse,novnc,python-ply,python-kitchen,python-daemon,python-websockify,livecd-tools,spice-html5,mom,python-IPy,python-ioprocess,ioprocess,rubygem-rgen
gpgcheck=1
gpgkey=https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-6
So why can engine depend on epel and vdsm can not?
engine and vdsm both can, and do, depend on epel. However, RHEV cannot.
Taking an epel package has a price of needing to ship it in
http://ftp.redhat.com/pub/redhat/rhev-m/3.x/SRPMS/ (see python-inotify
there, as a vdsm dependency).
In case of simplejson for el7, it simply does not worth it.
4:34 a.m.
New subject: [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
On Tue, Nov 11, 2014 at 05:24:43AM -0500, Francesco Romani wrote:
----- Original Message -----
> From: "Nir Soffer" <nsoffer(a)redhat.com>
> To: "Francesco Romani" <fromani(a)redhat.com>
> Cc: devel(a)ovirt.org
> Sent: Monday, November 10, 2014 1:21:53 PM
> Subject: Re: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance
on RHEL/Centos 6
>
> We have vdsm/lib/compat.py for this stuff.
>
> Then everyone needing json will do "from vdsm.compat import json"
Do you think is better to have an hard dependency on 'simplejson'
or a soft one like I showed above hidden on compat.py?
I think we agree that 'explicit is better than implicit' here, so... (see below).
I know ganeti has a hard dependency on simplejson for performance
reasons, so you wouldn't be the first virtualization project to do so :)
> Nice catch - using pure python encoder/decoder is indeed much
slower, strange
> that Python 2.6 does not include the c extension.
>
> I think we would like to use simplejson on all platforms, to make the build
> and support simpler.
...agreed, and posted http://gerrit.ovirt.org/#/c/35024/
2:08 p.m.
New subject: [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
It's well known.
The problem is that there is no simplejson in EL6\7
(unless the situation changed since I last checked).
I already did extensive profiling on the jsonrpc framework.
The patches [1] move us on par with XML-RPC for sequential requests and
*way* faster for concurrent commands.
There are thoughts about adding msgpack [2] as an optional encoding for
responses as it's much smaller and less cpu intensive to encode\decode
but we have bigger fish to fry ATM (patches welcome).
[1]
http://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:...
[2] http://msgpack.org/
----- Original Message -----
From: "Francesco Romani" <fromani(a)redhat.com>
To: devel(a)ovirt.org
Sent: Monday, November 10, 2014 10:02:44 AM
Subject: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
Hi everyone,
I was doing JSON-RPC investigation recently, running VDSM on RHEL6.{5,6},
and while some initial profile work, I discovered a (little) performance
degradation
about pure JSONRPC coding/encoding, *only* on the above platforms. Here's
why:
- the JSON codec module in the python stdlib is based on simplejson
- (thus) simplejson API is identical to stdlib's json module API
- python 2.6 JSON codec is (based on) simplejson 1.9
- python 2.7 JSON codec is (based on) simplejson 2.0.9, with significant
speedups. See
https://docs.python.org/2/whatsnew/2.7.html#new-and-improved-modules and
https://bugs.python.org/issue4136
- RHEL6.x/Centos 6.x includes python 2.6, so here (and only here) JSON codec
is unnecessarily slower.
- RHEL6.x (surely) and CentOS 6.x (need to check, don't see why not) includes
anyway simplejson 2.0.9 as external package
So, it seems to me that we can get python 2.7 speed even on stable platform
with
a five line patch:
=== cut here ===
diff --git a/lib/yajsonrpc/__init__.py b/lib/yajsonrpc/__init__.py
index b3fd590..6682fc3 100644
--- a/lib/yajsonrpc/__init__.py
+++ b/lib/yajsonrpc/__init__.py
@@ -12,7 +12,10 @@
# You should have received a copy of the GNU General Public
# License along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
-import json
+try:
+ import simplejson as json
+except ImportError
+ import json
import logging
from functools import partial
from Queue import Queue
=== cut here ===
Here's an excerpt of the effects of this patch, using an initial still rough
bencmark
- once again 100 VMs boot, is the beginning of every testing routine from
mine,
so nothing new here.
Let's consider two initial profiles. Please note that
1) the amount of calls is not the same (error from mine in profiling, need
more precise bench)
but still
2) using 2.0.9 module JSON just vanishes from the profile alltogether
Vanilla VDSM, stdlib json, top 20 expensive calls
ncalls tottime percall cumtime percall filename:lineno(function)
19/100 127.216 6.696 9.974 0.100
/usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run)
1555/1635 41.180 0.026 58.566 0.036
/usr/lib64/python2.6/threading.py:481(Thread.run)
11708/1553160 11.967 0.001 37.921 0.000
/usr/lib64/python2.6/copy.py:144(deepcopy)
1268613 6.154 0.000 9.209 0.000
/usr/lib64/python2.6/copy.py:261(_keep_alive)
4890/130093 6.149 0.001 37.596 0.000
/usr/lib64/python2.6/copy.py:251(_deepcopy_dict)
497858/2070078 5.224 0.000 11.102 0.000
/usr/lib64/python2.6/json/encoder.py:284(JSONEncoder._iterencode)
498440/1333474 4.187 0.000 10.368 0.000
/usr/lib64/python2.6/json/encoder.py:213(JSONEncoder._iterencode_dict)
12/100 4.115 0.343 4.130 0.041
/usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run)
43826 3.692 0.000 6.516 0.000
/usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler)
7345 3.657 0.000 11.047 0.002
/usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats)
9666/320508 3.568 0.000 3.572 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper)
274994/275000 3.339 0.000 5.593 0.000
/usr/lib64/python2.6/StringIO.py:208(StringIO.write)
90217 1.823 0.000 1.870 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__)
44025/44036 1.666 0.000 2.470 0.000
/usr/share/vdsm/storage/lvm.py:217(makeLV)
144212/144226 1.287 0.000 1.292 0.000
/usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr)
201 1.285 0.006 7.278 0.036
/usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs)
109880 1.235 0.000 1.283 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:281(Document._append_child)
57874 1.163 0.000 1.799 0.000
/usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata)
1130859 1.160 0.000 1.160 0.000
/usr/lib64/python2.6/copy.py:197(_deepcopy_atomic)
3021/3022 1.121 0.000 2.554 0.001
/usr/lib64/python2.6/inspect.py:247(getmembers)
patch applied, top 20 expensive calls
24/100 117.222 4.884 13.372 0.134
/usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run)
1209/1302 60.812 0.050 64.963 0.050
/usr/lib64/python2.6/threading.py:481(Thread.run)
9243/1235834 10.259 0.001 33.047 0.000
/usr/lib64/python2.6/copy.py:144(deepcopy)
13/100 7.007 0.539 7.281 0.073
/usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run)
4116/103669 5.536 0.001 32.779 0.000
/usr/lib64/python2.6/copy.py:251(_deepcopy_dict)
9670/320560 4.437 0.000 4.443 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper)
5653 3.232 0.001 9.443 0.002
/usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats)
43836 3.117 0.000 7.304 0.000
/usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler)
274967/275000 2.955 0.000 6.833 0.000
/usr/lib64/python2.6/StringIO.py:208(StringIO.write)
1100/28500 2.376 0.002 12.881 0.000
/usr/share/vdsm/virt/vmxml.py:489(Element.__hacked_writexml)
43745/44036 1.729 0.000 2.736 0.000
/usr/share/vdsm/storage/lvm.py:217(makeLV)
90217/90221 1.695 0.000 1.953 0.000
/usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__)
201 1.549 0.008 7.725 0.038
/usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs)
57888 1.441 0.000 2.216 0.000
/usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata)
2351/2354 1.262 0.001 2.750 0.001
/usr/lib64/python2.6/inspect.py:247(getmembers)
992794 1.232 0.000 8.027 0.000
/usr/lib64/python2.6/copy.py:261(_keep_alive)
4827 1.150 0.000 1.368 0.000
/usr/lib/python2.6/site-packages/mom/Policy/spark.py:211(Parser.buildState)
108040 1.119 0.000 1.119 0.000
/usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr)
336771/422122 1.106 0.000 2.575 0.000
/usr/share/vdsm/protocoldetector.py:108(MultiProtocolAcceptor._process_events)
883021 0.987 0.000 0.987 0.000
/usr/lib64/python2.6/copy.py:197(_deepcopy_atomic)
Indeed this is a microbenchmark, but my point is we *recover* some speed at
nearly
zero cost and risk.
Now, the question is:
Do we want this patch? Do we want it on master? On 3.5 branch? Only on
RHEL/Centos 6 platforms?
--
Francesco Romani
RedHat Engineering Virtualization R & D
Phone: 8261328
IRC: fromani
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
4:28 a.m.
New subject: [VDSM][JSON] jsonrpc coding/encoding performance on RHEL/Centos 6
On Sun, Nov 23, 2014 at 03:08:59PM -0500, Saggi Mizrahi wrote:
It's well known.
I'd drop the "well" - it was not known to me, and to others.
The problem is that there is no simplejson in EL6\7
(unless the situation changed since I last checked).
Francesco verified that RHEL6 does have simplejson. RHEL7 does not have
it (and does not need it as anxiously).
I already did extensive profiling on the jsonrpc framework.
The patches [1] move us on par with XML-RPC for sequential requests and
*way* faster for concurrent commands.
Thanks!
There are thoughts about adding msgpack [2] as an optional encoding for
responses as it's much smaller and less cpu intensive to encode\decode
but we have bigger fish to fry ATM (patches welcome).
[1]
http://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:...
[2] http://msgpack.org/
----- Original Message -----
> From: "Francesco Romani" <fromani(a)redhat.com>
> To: devel(a)ovirt.org
> Sent: Monday, November 10, 2014 10:02:44 AM
> Subject: [ovirt-devel] [VDSM][JSON] jsonrpc coding/encoding performance
on RHEL/Centos 6
>
> Hi everyone,
>
> I was doing JSON-RPC investigation recently, running VDSM on RHEL6.{5,6},
> and while some initial profile work, I discovered a (little) performance
> degradation
> about pure JSONRPC coding/encoding, *only* on the above platforms. Here's
> why:
>
> - the JSON codec module in the python stdlib is based on simplejson
> - (thus) simplejson API is identical to stdlib's json module API
> - python 2.6 JSON codec is (based on) simplejson 1.9
> - python 2.7 JSON codec is (based on) simplejson 2.0.9, with significant
> speedups. See
> https://docs.python.org/2/whatsnew/2.7.html#new-and-improved-modules and
> https://bugs.python.org/issue4136
> - RHEL6.x/Centos 6.x includes python 2.6, so here (and only here) JSON codec
> is unnecessarily slower.
> - RHEL6.x (surely) and CentOS 6.x (need to check, don't see why not) includes
> anyway simplejson 2.0.9 as external package
>
> So, it seems to me that we can get python 2.7 speed even on stable platform
> with
> a five line patch:
>
> === cut here ===
> diff --git a/lib/yajsonrpc/__init__.py b/lib/yajsonrpc/__init__.py
> index b3fd590..6682fc3 100644
> --- a/lib/yajsonrpc/__init__.py
> +++ b/lib/yajsonrpc/__init__.py
> @@ -12,7 +12,10 @@
> # You should have received a copy of the GNU General Public
> # License along with this program; if not, write to the Free Software
> # Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> -import json
> +try:
> + import simplejson as json
> +except ImportError
> + import json
> import logging
> from functools import partial
> from Queue import Queue
> === cut here ===
>
> Here's an excerpt of the effects of this patch, using an initial still rough
> bencmark
> - once again 100 VMs boot, is the beginning of every testing routine from
> mine,
> so nothing new here.
>
> Let's consider two initial profiles. Please note that
> 1) the amount of calls is not the same (error from mine in profiling, need
> more precise bench)
> but still
> 2) using 2.0.9 module JSON just vanishes from the profile alltogether
>
> Vanilla VDSM, stdlib json, top 20 expensive calls
>
> ncalls tottime percall cumtime percall filename:lineno(function)
> 19/100 127.216 6.696 9.974 0.100
> /usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run)
> 1555/1635 41.180 0.026 58.566 0.036
> /usr/lib64/python2.6/threading.py:481(Thread.run)
> 11708/1553160 11.967 0.001 37.921 0.000
> /usr/lib64/python2.6/copy.py:144(deepcopy)
> 1268613 6.154 0.000 9.209 0.000
> /usr/lib64/python2.6/copy.py:261(_keep_alive)
> 4890/130093 6.149 0.001 37.596 0.000
> /usr/lib64/python2.6/copy.py:251(_deepcopy_dict)
> 497858/2070078 5.224 0.000 11.102 0.000
> /usr/lib64/python2.6/json/encoder.py:284(JSONEncoder._iterencode)
> 498440/1333474 4.187 0.000 10.368 0.000
> /usr/lib64/python2.6/json/encoder.py:213(JSONEncoder._iterencode_dict)
> 12/100 4.115 0.343 4.130 0.041
> /usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run)
> 43826 3.692 0.000 6.516 0.000
>
/usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler)
> 7345 3.657 0.000 11.047 0.002
> /usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats)
> 9666/320508 3.568 0.000 3.572 0.000
> /usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper)
> 274994/275000 3.339 0.000 5.593 0.000
> /usr/lib64/python2.6/StringIO.py:208(StringIO.write)
> 90217 1.823 0.000 1.870 0.000
> /usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__)
> 44025/44036 1.666 0.000 2.470 0.000
> /usr/share/vdsm/storage/lvm.py:217(makeLV)
> 144212/144226 1.287 0.000 1.292 0.000
> /usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr)
> 201 1.285 0.006 7.278 0.036
> /usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs)
> 109880 1.235 0.000 1.283 0.000
> /usr/lib64/python2.6/xml/dom/minidom.py:281(Document._append_child)
> 57874 1.163 0.000 1.799 0.000
>
/usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata)
> 1130859 1.160 0.000 1.160 0.000
> /usr/lib64/python2.6/copy.py:197(_deepcopy_atomic)
> 3021/3022 1.121 0.000 2.554 0.001
> /usr/lib64/python2.6/inspect.py:247(getmembers)
>
>
> patch applied, top 20 expensive calls
>
> 24/100 117.222 4.884 13.372 0.134
> /usr/lib/python2.6/site-packages/mom/GuestMonitor.py:51(GuestMonitor.run)
> 1209/1302 60.812 0.050 64.963 0.050
> /usr/lib64/python2.6/threading.py:481(Thread.run)
> 9243/1235834 10.259 0.001 33.047 0.000
> /usr/lib64/python2.6/copy.py:144(deepcopy)
> 13/100 7.007 0.539 7.281 0.073
> /usr/share/vdsm/virt/sampling.py:424(VmStatsThread.run)
> 4116/103669 5.536 0.001 32.779 0.000
> /usr/lib64/python2.6/copy.py:251(_deepcopy_dict)
> 9670/320560 4.437 0.000 4.443 0.000
> /usr/lib64/python2.6/xml/dom/minidom.py:305(_get_elements_by_tagName_helper)
> 5653 3.232 0.001 9.443 0.002
> /usr/share/vdsm/virt/vm.py:2264(Vm._getRunningVmStats)
> 43836 3.117 0.000 7.304 0.000
>
/usr/lib64/python2.6/xml/dom/expatbuilder.py:743(ExpatBuilderNS.start_element_handler)
> 274967/275000 2.955 0.000 6.833 0.000
> /usr/lib64/python2.6/StringIO.py:208(StringIO.write)
> 1100/28500 2.376 0.002 12.881 0.000
> /usr/share/vdsm/virt/vmxml.py:489(Element.__hacked_writexml)
> 43745/44036 1.729 0.000 2.736 0.000
> /usr/share/vdsm/storage/lvm.py:217(makeLV)
> 90217/90221 1.695 0.000 1.953 0.000
> /usr/lib64/python2.6/xml/dom/minidom.py:349(Attr.__init__)
> 201 1.549 0.008 7.725 0.038
> /usr/share/vdsm/storage/lvm.py:411(LVMCache._reloadlvs)
> 57888 1.441 0.000 2.216 0.000
>
/usr/lib64/python2.6/xml/dom/expatbuilder.py:274(ExpatBuilderNS.character_data_handler_cdata)
> 2351/2354 1.262 0.001 2.750 0.001
> /usr/lib64/python2.6/inspect.py:247(getmembers)
> 992794 1.232 0.000 8.027 0.000
> /usr/lib64/python2.6/copy.py:261(_keep_alive)
> 4827 1.150 0.000 1.368 0.000
> /usr/lib/python2.6/site-packages/mom/Policy/spark.py:211(Parser.buildState)
> 108040 1.119 0.000 1.119 0.000
> /usr/lib/python2.6/site-packages/vdsm/utils.py:285(convertToStr)
> 336771/422122 1.106 0.000 2.575 0.000
> /usr/share/vdsm/protocoldetector.py:108(MultiProtocolAcceptor._process_events)
> 883021 0.987 0.000 0.987 0.000
> /usr/lib64/python2.6/copy.py:197(_deepcopy_atomic)
>
>
> Indeed this is a microbenchmark, but my point is we *recover* some speed at
> nearly
> zero cost and risk.
>
> Now, the question is:
> Do we want this patch? Do we want it on master? On 3.5 branch? Only on
> RHEL/Centos 6 platforms?
>
> --
> Francesco Romani
> RedHat Engineering Virtualization R & D
> Phone: 8261328
> IRC: fromani
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
3464
days inactive
3478
days old
13 comments
7 participants
participants (7)
-
Dan Kenigsberg -
Ewoud Kohl van Wijngaarden -
Francesco Romani -
Nir Soffer -
Piotr Kliczewski -
Saggi Mizrahi -
Sven Kieske