[Users] Python SDK list vms slow, excessive cpu usage

[Michael Pasternak]

Hi Sander,

This is a known issue caused by generateDS python bindings we use,
it's extremely slow in python<->xml marshalling, and unable to recognize
cyclic referencing in the objects,

i'm planning to upgrade in 3.4 from 2.9a to 2.12, if it won't help, we may
consider other options.

On 12/17/2013 04:04 PM, Sander Grendelman wrote:
> Fetching the list of vms through the API with the python SDK
> takes several seconds[1] with 100% cpu usage in the python script.
> 
> When I look at the engine log there is only one (fast) fetch from
> the API. The rest of the time is spent in the SDK processing a
> relatively small bit of XML data (only 26 VMs in my environment).
> 
> This seems an excessive amount of CPU for processing ~6KB of XML.
> 
> I've included some sample code [2] and output [3].
> Attached is the cProfile output for this call and a visualization.
> 
> [1] ~6,5 seconds on an oVirt VM with 1 vcpu on older hardware,
>     ~3,5 seconds on a fast physical machine with an i5 cpu.
> 
> [2] Sample code, add your own url/credentials/certificate:
> 
> #!/usr/bin/python
> """Get ovirt VM names"""
> 
> import time
> 
> from ovirtsdk.api import API
> from ovirtsdk.xml import params
> 
> def print_elapsed(logString):
>     elapsed = time.time() - startTime
>     print "%fs %s" % (elapsed, logString)
> 
> startTime = time.time()
> 
> print_elapsed("before connect")
> api = API(
>         url="https://my.engine.url/api",
>         username="user at domain",
>         password="password",
>         ca_file="./ca.crt",
>     )
> print_elapsed("after connect")
> 
> print_elapsed("before fetch list")
> vmList = api.vms.list()
> print_elapsed("after fetch list")
> 
> print_elapsed("before use list")
> count = 0
> for vm in vmList:
>   vmName = vm.get_name()
>   count += 1
> 
> print "\t number of VMs %d" % count
> 
> [3] Example of output:
> 
> [grendelmans at bungee src]$ /usr/bin/time ./vm_list_test.py
> 0.000003s before connect
> 0.338248s after connect
> 0.338299s before fetch list
> 6.795967s after fetch list
> 6.796041s before use list
>      number of VMs 26
> 6.796181s after use list
> 7.28user 0.05system 0:07.64elapsed 96%CPU (0avgtext+0avgdata 84624maxresident)k
> 0inputs+0outputs (0major+5605minor)pagefaults 0swaps
> [grendelmans at bungee src]$
> 
> 
> 
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
> 


-- 

Michael Pasternak
RedHat, ENG-Virtualization R&D