[ovirt-users] [ovirt-devel] ovirt-engine-sdk-python too slow

Tue Nov 24 21:49:34 UTC 2015

On Tue, Nov 24, 2015 at 3:49 PM, John Hunter <zhjwpku at gmail.com> wrote:

>
>
> On Tue, Nov 24, 2015 at 9:15 PM, Juan Hernández <jhernand at redhat.com>
> wrote:
>
>> On 11/24/2015 01:40 PM, John Hunter wrote:
>> > Hi,
>> >
>> > On Tue, Nov 24, 2015 at 5:18 PM, Oved Ourfali <oourfali at redhat.com
>> > <mailto:oourfali at redhat.com>> wrote:
>> >
>> >     Hi
>> >
>> >     I discussed it with Juan (cc-ed).
>> >
>> >     There used to be a bug in the JDBC authenticion extension that
>> >     artificially delayed RESTAPI responses by 5 seconds:
>> >
>> >       brute force prevention login delay should not be applied to
>> successful
>> >     login requests
>> >       https://bugzilla.redhat.com/1255814
>> >
>> >     That matches the description of the issue, but in theory it has been
>> >     fixed. I would suggest him to check that he is using the right
>> version
>> >     of the extension.
>> >
>> > I did not use the extension ovirt-engine-extension-aaa-jdbc, and I don't
>> > think this bug matches my problem, because even there is only one line
>> > in the python script, it still cost like 3 seconds, I don't think this
>> is a
>> > reasonable time as when I import other package, it cost almost no time.
>> >
>> > Can you explain why this import line costs so much time?
>> >
>>
>> If you are using 3.6 then you are using ovirt-engine-extension-aaa-jdbc,
>> as it is enabled by default, but looks like it isn't related to your
>> problem.
>>
>> That line takes a long time to execute because it has to process two
>> large Python modules: the "params" module that contains a class per each
>> type used by the API (393 classes) and the "brokers" module that
>> contains a class per each resource used by the API (358 classes). That
>> makes a total of 751 classes. In my environment it takes 0.9 seconds,
>> approx. You may want to use the python profile in your environment and
>> share the results:
>>
>> $ cat > profile.py <<.
>> import cProfile
>> cProfile.run("from ovirtsdk.api import API")
>> .
>>
>> $ python profile.py
>>
>> I won't be surprised to see this taking those 3 seconds in a slower
>> environment.
>>
>> But even if this takes those 3 seconds it shouldn't be a big problem,
>> because you shouldn't be running that "from ... import ..." line
>> frequently. Your program should do this once only, when it starts.
>>
>> Assume that I have two functions to implement, one is to list all the vms
> belong
> to the user, and the other is to retrieve one vm's virt-viewer connection
> file, as
> far as I can see, I have to write two python scripts and import the
> ovirtsdk.api in both
> scripts, each script has to take the 3 seconds :(
>

No, you have two functions, which can be in the same module, or in
different modules, depending on how you want to organize your code.

Python imports are done only once, on the first time you import a module
anywhere. The module is stored in sys.modules, and the next import fetch
the module object from sys.modules.

Can you share a profile of a simple script importing the Python sdk?

Do:

    python -m cProfile -o myscript.prof myscript.py

The profile will be stored in myscript.prof.

For viewing the profile, I recommend to use the profile-stats tool:
https://github.com/oVirt/vdsm/blob/master/contrib/profile-stats

>
> How can I run the "from ... import ..." just once ?
>
>
>> >
>> >     In addition we also know that retrieving large lists of objects
>> with the
>> >     SDK is slow:
>> >
>> >        [RFE][performance] - generate large scale list running to slow.
>> >        https://bugzilla.redhat.com/1221238
>> >
>> >     We don't have a solution for that yet.
>> >
>> >     CC-ing Juan in case you have additional questions.
>> >
>> >
>> >     On Mon, Nov 23, 2015 at 11:27 AM, John Hunter <zhjwpku at gmail.com
>> >     <mailto:zhjwpku at gmail.com>> wrote:
>> >
>> >         Hi guys,
>> >
>> >         I am using the ovirt-engine-sdk-python to communicate with the
>> >         ovirt-engine,
>> >         I am ok to list the vms but the processing time is too long,
>> >         like 4.5 seconds,
>> >         and this line:
>> >             from ovirtsdk.api import API
>> >         take almost 3 seconds.
>> >
>> >         This seems a little bit longer than I expected it to be, so I am
>> >         asking is there
>> >         a quicker way to communicate with the ovirt-engine?
>> >
>>
>> --
>> Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
>> 3ºD, 28016 Madrid, Spain
>> Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L.
>>
>
>
>
> --
> Best regards
> Junwang Zhao
> Department of Computer Science &Technology
> Peking University
> Beijing, 100871, PRC
>
> _______________________________________________
> Devel mailing list
> Devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20151124/1a690079/attachment-0001.html>