[ovirt-devel] XML benchmarks

[Nir Soffer]

----- Original Message -----
> From: "Francesco Romani" <fromani at redhat.com>
> To: devel at ovirt.org
> Sent: Friday, June 27, 2014 3:30:14 PM
> Subject: [ovirt-devel] XML benchmarks
> 
> Hi,
> 
> Due to the recent discussion (http://gerrit.ovirt.org/#/c/28712/), and as
> part
> of the ongoing focus on scalability and performances
> (http://gerrit.ovirt.org/#/c/17694/ and many others),
> 
> I took the chance to do a very quick and dirty bench to see how it really
> cost
> to do XML processing in sampling threads (thanks to Nir for the kickstart!),
> and,
> in general, how much the XML processing costs.
> 
> Please find attached the test script and the example XML
> (real one made by VDSM master on my RHEL6.5 box).
> 
> On my laptop:
> 
> $ lscpu
> Architecture:          x86_64
> CPU op-mode(s):        32-bit, 64-bit
> Byte Order:            Little Endian
> CPU(s):                4
> On-line CPU(s) list:   0-3
> Thread(s) per core:    2
> Core(s) per socket:    2
> Socket(s):             1
> NUMA node(s):          1
> Vendor ID:             GenuineIntel
> CPU family:            6
> Model:                 58
> Model name:            Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz
> Stepping:              9
> CPU MHz:               1359.375
> CPU max MHz:           3600.0000
> CPU min MHz:           1200.0000
> BogoMIPS:              5786.91
> Virtualization:        VT-x
> L1d cache:             32K
> L1i cache:             32K
> L2 cache:              256K
> L3 cache:              4096K
> NUMA node0 CPU(s):     0-3
> 
> 8 GiBs of RAM, running GNOME desktop and the usual development stuff
> 
> xmlbench.py linuxvm1.xml MODE 300
> 
> MODE is either 'md' (minidom) or 'cet' (cElementTree).
> This will run $NUMTHREADS threads fast and loose without synchronization.
> We can actually have this behaviour if a customer just mass start VMs.
> In general I expect some clustering of the sampling activity, not a nice
> evenly interleaved
> time sequence.
> 
> CPU measurement: just opened a terminal and run 'htop' on it.
> CPU profile: clustered around the sampling interval. Usage negligible most of
> time, peak on sampling as shown below
> 
> 300 VMs
> minidom: ~38% CPU
> cElementTree: ~5% CPU

What is 38% - (38% of one core? how may cores are on the machine?)

> 
> 500 VMs
> minidom: ~48% CPU
> cElementTree: ~6% CPU
> 
> 1000 VMs
> python thread error :)
> 
>   File "/usr/lib64/python2.7/threading.py", line 746, in start
>     _start_new_thread(self.__bootstrap, ())
> thread.error: can't start new thread
> 
> 
> I think this is another proof (if we need more of them) that
> * we _really need_ to move away from the 1 thread per VM model ->
> http://gerrit.ovirt.org/#/c/29189/ and friends! Let's fire up the
> discussion!
> * we should move to cElementTree anyway in the near future: faster
> processing, scales better, nicer API.
>   It is also a pet peeve of mine, I do have some patches floating but we need
>   still some preparation work in the virt package.

Seeing this load created by parsing libvirt xml every 15 seconds, I think
we should consider decreasing the sample rate suggested in
http://gerrit.ovirt.org/28712 Or collecting the data in another way.

Nir