Re: [ovirt-users] [ovirt-devel] ovirt-engine-sdk-python too slow

Hi Nir, Attachment is my script and its profile. Thanks a lot about your help! On Wed, Nov 25, 2015 at 5:49 AM, Nir Soffer <nsoffer(a)redhat.com&gt; wrote: > On Tue, Nov 24, 2015 at 3:49 PM, John Hunter <zhjwpku(a)gmail.com&gt; wrote: > >> >> >> On Tue, Nov 24, 2015 at 9:15 PM, Juan Hernández <jhernand(a)redhat.com&gt; >> wrote: >> >>> On 11/24/2015 01:40 PM, John Hunter wrote: >>> > Hi, >>> > >>> > On Tue, Nov 24, 2015 at 5:18 PM, Oved Ourfali <oourfali(a)redhat.com >>> > <mailto:oourfali@redhat.com>> wrote: >>> > >>> > Hi >>> > >>> > I discussed it with Juan (cc-ed). >>> > >>> > There used to be a bug in the JDBC authenticion extension that >>> > artificially delayed RESTAPI responses by 5 seconds: >>> > >>> > brute force prevention login delay should not be applied to >>> successful >>> > login requests >>> > https://bugzilla.redhat.com/1255814 >>> > >>> > That matches the description of the issue, but in theory it has >>> been >>> > fixed. I would suggest him to check that he is using the right >>> version >>> > of the extension. >>> > >>> > I did not use the extension ovirt-engine-extension-aaa-jdbc, and I >>> don't >>> > think this bug matches my problem, because even there is only one line >>> > in the python script, it still cost like 3 seconds, I don't think >>> this is a >>> > reasonable time as when I import other package, it cost almost no >>> time. >>> > >>> > Can you explain why this import line costs so much time? >>> > >>> >>> If you are using 3.6 then you are using ovirt-engine-extension-aaa-jdbc, >>> as it is enabled by default, but looks like it isn't related to your >>> problem. >>> >>> That line takes a long time to execute because it has to process two >>> large Python modules: the "params" module that contains a class per each >>> type used by the API (393 classes) and the "brokers" module that >>> contains a class per each resource used by the API (358 classes). That >>> makes a total of 751 classes. In my environment it takes 0.9 seconds, >>> approx. You may want to use the python profile in your environment and >>> share the results: >>> >>> $ cat > profile.py <<. >>> import cProfile >>> cProfile.run("from ovirtsdk.api import API") >>> . >>> >>> $ python profile.py >>> >>> I won't be surprised to see this taking those 3 seconds in a slower >>> environment. >>> >>> But even if this takes those 3 seconds it shouldn't be a big problem, >>> because you shouldn't be running that "from ... import ..." line >>> frequently. Your program should do this once only, when it starts. >>> >>> Assume that I have two functions to implement, one is to list all the >> vms belong >> to the user, and the other is to retrieve one vm's virt-viewer >> connection file, as >> far as I can see, I have to write two python scripts and import the >> ovirtsdk.api in both >> scripts, each script has to take the 3 seconds :( >> > > No, you have two functions, which can be in the same module, or in > different modules, depending on how you want to organize your code. > > Python imports are done only once, on the first time you import a module > anywhere. The module is stored in sys.modules, and the next import fetch > the module object from sys.modules. > > Can you share a profile of a simple script importing the Python sdk? > > Do: > > python -m cProfile -o myscript.prof myscript.py > > The profile will be stored in myscript.prof. > > For viewing the profile, I recommend to use the profile-stats tool: > https://github.com/oVirt/vdsm/blob/master/contrib/profile-stats > > >> >> How can I run the "from ... import ..." just once ? >> >> >>> > >>> > In addition we also know that retrieving large lists of objects >>> with the >>> > SDK is slow: >>> > >>> > [RFE][performance] - generate large scale list running to slow. >>> > https://bugzilla.redhat.com/1221238 >>> > >>> > We don't have a solution for that yet. >>> > >>> > CC-ing Juan in case you have additional questions. >>> > >>> > >>> > On Mon, Nov 23, 2015 at 11:27 AM, John Hunter <zhjwpku(a)gmail.com >>> > <mailto:zhjwpku@gmail.com>> wrote: >>> > >>> > Hi guys, >>> > >>> > I am using the ovirt-engine-sdk-python to communicate with the >>> > ovirt-engine, >>> > I am ok to list the vms but the processing time is too long, >>> > like 4.5 seconds, >>> > and this line: >>> > from ovirtsdk.api import API >>> > take almost 3 seconds. >>> > >>> > This seems a little bit longer than I expected it to be, so I >>> am >>> > asking is there >>> > a quicker way to communicate with the ovirt-engine? >>> > >>> >>> -- >>> Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta >>> 3ºD, 28016 Madrid, Spain >>> Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat >>> S.L. >>> >> >> >> >> -- >> Best regards >> Junwang Zhao >> Department of Computer Science &Technology >> Peking University >> Beijing, 100871, PRC >> >> _______________________________________________ >> Devel mailing list >> Devel(a)ovirt.org >> http://lists.ovirt.org/mailman/listinfo/devel >> > > -- Best regards Junwang Zhao Department of Computer Science &Technology Peking University Beijing, 100871, PRC