[ovirt-users] [ovirt-devel] ovirt-engine-sdk-python too slow

[John Hunter]

On Tue, Nov 24, 2015 at 9:56 PM, Juan Hernández <jhernand at redhat.com> wrote:

> On 11/24/2015 02:49 PM, John Hunter wrote:
> >
> >
> > On Tue, Nov 24, 2015 at 9:15 PM, Juan Hernández <jhernand at redhat.com
> > <mailto:jhernand at redhat.com>> wrote:
> >
> >     On 11/24/2015 01:40 PM, John Hunter wrote:
> >     > Hi,
> >     >
> >     > On Tue, Nov 24, 2015 at 5:18 PM, Oved Ourfali <oourfali at redhat.com
> <mailto:oourfali at redhat.com>
> >     > <mailto:oourfali at redhat.com <mailto:oourfali at redhat.com>>> wrote:
> >     >
> >     >     Hi
> >     >
> >     >     I discussed it with Juan (cc-ed).
> >     >
> >     >     There used to be a bug in the JDBC authenticion extension that
> >     >     artificially delayed RESTAPI responses by 5 seconds:
> >     >
> >     >       brute force prevention login delay should not be applied to
> successful
> >     >     login requests
> >     >       https://bugzilla.redhat.com/1255814
> >     >
> >     >     That matches the description of the issue, but in theory it
> has been
> >     >     fixed. I would suggest him to check that he is using the right
> version
> >     >     of the extension.
> >     >
> >     > I did not use the extension ovirt-engine-extension-aaa-jdbc, and I
> don't
> >     > think this bug matches my problem, because even there is only one
> line
> >     > in the python script, it still cost like 3 seconds, I don't think
> this is a
> >     > reasonable time as when I import other package, it cost almost no
> time.
> >     >
> >     > Can you explain why this import line costs so much time?
> >     >
> >
> >     If you are using 3.6 then you are using
> ovirt-engine-extension-aaa-jdbc,
> >     as it is enabled by default, but looks like it isn't related to your
> >     problem.
> >
> >     That line takes a long time to execute because it has to process two
> >     large Python modules: the "params" module that contains a class per
> each
> >     type used by the API (393 classes) and the "brokers" module that
> >     contains a class per each resource used by the API (358 classes).
> That
> >     makes a total of 751 classes. In my environment it takes 0.9 seconds,
> >     approx. You may want to use the python profile in your environment
> and
> >     share the results:
> >
> >     $ cat > profile.py <<.
> >     import cProfile
> >     cProfile.run("from ovirtsdk.api import API")
> >     .
> >
> >     $ python profile.py
> >
> >     I won't be surprised to see this taking those 3 seconds in a slower
> >     environment.
> >
> >     But even if this takes those 3 seconds it shouldn't be a big problem,
> >     because you shouldn't be running that "from ... import ..." line
> >     frequently. Your program should do this once only, when it starts.
> >
> > Assume that I have two functions to implement, one is to list all the
> > vms belong
> > to the user, and the other is to retrieve one vm's virt-viewer
> > connection file, as
> > far as I can see, I have to write two python scripts and import the
> > ovirtsdk.api in both
> > scripts, each script has to take the 3 seconds :(
> >
> > How can I run the "from ... import ..." just once ?
> >
>
> I don't know what technology or tools are you using to write that
> program, but if you are using Python then you don't need to import it
> twice.
>
> Yeah, I am just using Python scripts to access the RESTAPI, since I am
not skillful in python, I might have asked some stupid question.

I will try to learn more about Python.

Anyway, thanks for your patience.


> If for whatever the reason you decide to call external python scripts
> from another program then you will have to pay the price of the startup
> of the Python SDK. If that is unacceptable because of performance then
> you should look for a different way to access the RESTAPI, like sending
> XML or JSON directly, or using the Java SDK, or rbovirt, it all depends
> on the technology that you are using for your application.
>
> >
> >     >
> >     >     In addition we also know that retrieving large lists of
> objects with the
> >     >     SDK is slow:
> >     >
> >     >        [RFE][performance] - generate large scale list running to
> slow.
> >     >        https://bugzilla.redhat.com/1221238
> >     >
> >     >     We don't have a solution for that yet.
> >     >
> >     >     CC-ing Juan in case you have additional questions.
> >     >
> >     >
> >     >     On Mon, Nov 23, 2015 at 11:27 AM, John Hunter <
> zhjwpku at gmail.com <mailto:zhjwpku at gmail.com>
> >     >     <mailto:zhjwpku at gmail.com <mailto:zhjwpku at gmail.com>>> wrote:
> >     >
> >     >         Hi guys,
> >     >
> >     >         I am using the ovirt-engine-sdk-python to communicate with
> the
> >     >         ovirt-engine,
> >     >         I am ok to list the vms but the processing time is too
> long,
> >     >         like 4.5 seconds,
> >     >         and this line:
> >     >             from ovirtsdk.api import API
> >     >         take almost 3 seconds.
> >     >
> >     >         This seems a little bit longer than I expected it to be,
> so I am
> >     >         asking is there
> >     >         a quicker way to communicate with the ovirt-engine?
> >     >
> >
>
> --
> Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
> 3ºD, 28016 Madrid, Spain
> Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L.
>



-- 
Best regards
Junwang Zhao
Department of Computer Science &Technology
Peking University
Beijing, 100871, PRC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20151125/3671e287/attachment-0001.html>