[ovirt-users] [ovirt-devel] ovirt-engine-sdk-python too slow
John Hunter
zhjwpku at gmail.com
Wed Nov 25 05:49:47 UTC 2015
Hi Nir,
Attachment is my script and its profile.
Thanks a lot about your help!
On Wed, Nov 25, 2015 at 5:49 AM, Nir Soffer <nsoffer at redhat.com> wrote:
> On Tue, Nov 24, 2015 at 3:49 PM, John Hunter <zhjwpku at gmail.com> wrote:
>
>>
>>
>> On Tue, Nov 24, 2015 at 9:15 PM, Juan Hernández <jhernand at redhat.com>
>> wrote:
>>
>>> On 11/24/2015 01:40 PM, John Hunter wrote:
>>> > Hi,
>>> >
>>> > On Tue, Nov 24, 2015 at 5:18 PM, Oved Ourfali <oourfali at redhat.com
>>> > <mailto:oourfali at redhat.com>> wrote:
>>> >
>>> > Hi
>>> >
>>> > I discussed it with Juan (cc-ed).
>>> >
>>> > There used to be a bug in the JDBC authenticion extension that
>>> > artificially delayed RESTAPI responses by 5 seconds:
>>> >
>>> > brute force prevention login delay should not be applied to
>>> successful
>>> > login requests
>>> > https://bugzilla.redhat.com/1255814
>>> >
>>> > That matches the description of the issue, but in theory it has
>>> been
>>> > fixed. I would suggest him to check that he is using the right
>>> version
>>> > of the extension.
>>> >
>>> > I did not use the extension ovirt-engine-extension-aaa-jdbc, and I
>>> don't
>>> > think this bug matches my problem, because even there is only one line
>>> > in the python script, it still cost like 3 seconds, I don't think this
>>> is a
>>> > reasonable time as when I import other package, it cost almost no time.
>>> >
>>> > Can you explain why this import line costs so much time?
>>> >
>>>
>>> If you are using 3.6 then you are using ovirt-engine-extension-aaa-jdbc,
>>> as it is enabled by default, but looks like it isn't related to your
>>> problem.
>>>
>>> That line takes a long time to execute because it has to process two
>>> large Python modules: the "params" module that contains a class per each
>>> type used by the API (393 classes) and the "brokers" module that
>>> contains a class per each resource used by the API (358 classes). That
>>> makes a total of 751 classes. In my environment it takes 0.9 seconds,
>>> approx. You may want to use the python profile in your environment and
>>> share the results:
>>>
>>> $ cat > profile.py <<.
>>> import cProfile
>>> cProfile.run("from ovirtsdk.api import API")
>>> .
>>>
>>> $ python profile.py
>>>
>>> I won't be surprised to see this taking those 3 seconds in a slower
>>> environment.
>>>
>>> But even if this takes those 3 seconds it shouldn't be a big problem,
>>> because you shouldn't be running that "from ... import ..." line
>>> frequently. Your program should do this once only, when it starts.
>>>
>>> Assume that I have two functions to implement, one is to list all the
>> vms belong
>> to the user, and the other is to retrieve one vm's virt-viewer connection
>> file, as
>> far as I can see, I have to write two python scripts and import the
>> ovirtsdk.api in both
>> scripts, each script has to take the 3 seconds :(
>>
>
> No, you have two functions, which can be in the same module, or in
> different modules, depending on how you want to organize your code.
>
> Python imports are done only once, on the first time you import a module
> anywhere. The module is stored in sys.modules, and the next import fetch
> the module object from sys.modules.
>
> Can you share a profile of a simple script importing the Python sdk?
>
> Do:
>
> python -m cProfile -o myscript.prof myscript.py
>
> The profile will be stored in myscript.prof.
>
> For viewing the profile, I recommend to use the profile-stats tool:
> https://github.com/oVirt/vdsm/blob/master/contrib/profile-stats
>
>
>>
>> How can I run the "from ... import ..." just once ?
>>
>>
>>> >
>>> > In addition we also know that retrieving large lists of objects
>>> with the
>>> > SDK is slow:
>>> >
>>> > [RFE][performance] - generate large scale list running to slow.
>>> > https://bugzilla.redhat.com/1221238
>>> >
>>> > We don't have a solution for that yet.
>>> >
>>> > CC-ing Juan in case you have additional questions.
>>> >
>>> >
>>> > On Mon, Nov 23, 2015 at 11:27 AM, John Hunter <zhjwpku at gmail.com
>>> > <mailto:zhjwpku at gmail.com>> wrote:
>>> >
>>> > Hi guys,
>>> >
>>> > I am using the ovirt-engine-sdk-python to communicate with the
>>> > ovirt-engine,
>>> > I am ok to list the vms but the processing time is too long,
>>> > like 4.5 seconds,
>>> > and this line:
>>> > from ovirtsdk.api import API
>>> > take almost 3 seconds.
>>> >
>>> > This seems a little bit longer than I expected it to be, so I
>>> am
>>> > asking is there
>>> > a quicker way to communicate with the ovirt-engine?
>>> >
>>>
>>> --
>>> Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
>>> 3ºD, 28016 Madrid, Spain
>>> Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L.
>>>
>>
>>
>>
>> --
>> Best regards
>> Junwang Zhao
>> Department of Computer Science &Technology
>> Peking University
>> Beijing, 100871, PRC
>>
>> _______________________________________________
>> Devel mailing list
>> Devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/devel
>>
>
>
--
Best regards
Junwang Zhao
Department of Computer Science &Technology
Peking University
Beijing, 100871, PRC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20151125/fb343260/attachment-0001.html>
-------------- next part --------------
#!/usr/bin/python
import time
tt = time.time()
print tt
from ovirtsdk.api import API
from ovirtsdk.xml import params
import time
tt = time.time()
print tt
def main():
URL = 'https://192.168.1.112:443/api'
USERNAME = 'admin at internal'
PASSWORD = 'mprc'
api = API(url=URL, username=USERNAME, password=PASSWORD, insecure=True)
vm = api.vms.get(name="ubuntu14.04")
print vm.id
#vm_list = api.vms.list()
#for vm in vm_list:
# print vm.name
api.disconnect()
if __name__ == '__main__':
main()
-------------- next part --------------
A non-text attachment was scrubbed...
Name: myscript.prof
Type: application/octet-stream
Size: 405694 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20151125/fb343260/attachment-0001.obj>
More information about the Users
mailing list