Re: [ovirt-devel] XML benchmarks

From: "Francesco Romani" <fromani(a)redhat.com&gt; To: devel(a)ovirt.org Sent: Friday, June 27, 2014 3:30:14 PM Subject: [ovirt-devel] XML benchmarks Hi, Due to the recent discussion (http://gerrit.ovirt.org/#/c/28712/), and as part of the ongoing focus on scalability and performances (http://gerrit.ovirt.org/#/c/17694/ and many others), I took the chance to do a very quick and dirty bench to see how it really cost to do XML processing in sampling threads (thanks to Nir for the kickstart!), and, in general, how much the XML processing costs. Please find attached the test script and the example XML (real one made by VDSM master on my RHEL6.5 box). On my laptop: $ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Thread(s) per core: 2 Core(s) per socket: 2 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 58 Model name: Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz Stepping: 9 CPU MHz: 1359.375 CPU max MHz: 3600.0000 CPU min MHz: 1200.0000 BogoMIPS: 5786.91 Virtualization: VT-x L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 4096K NUMA node0 CPU(s): 0-3 8 GiBs of RAM, running GNOME desktop and the usual development stuff xmlbench.py linuxvm1.xml MODE 300 MODE is either 'md' (minidom) or 'cet' (cElementTree). This will run $NUMTHREADS threads fast and loose without synchronization. We can actually have this behaviour if a customer just mass start VMs. In general I expect some clustering of the sampling activity, not a nice evenly interleaved time sequence. CPU measurement: just opened a terminal and run 'htop' on it. CPU profile: clustered around the sampling interval. Usage negligible most of time, peak on sampling as shown below 300 VMs minidom: ~38% CPU cElementTree: ~5% CPU

500 VMs minidom: ~48% CPU cElementTree: ~6% CPU 1000 VMs python thread error :) File "/usr/lib64/python2.7/threading.py", line 746, in start _start_new_thread(self.__bootstrap, ()) thread.error: can't start new thread I think this is another proof (if we need more of them) that * we _really need_ to move away from the 1 thread per VM model -> http://gerrit.ovirt.org/#/c/29189/ and friends! Let's fire up the discussion! * we should move to cElementTree anyway in the near future: faster processing, scales better, nicer API. It is also a pet peeve of mine, I do have some patches floating but we need still some preparation work in the virt package.