<div dir="ltr">$ ./profile-stats -c myscript.prof<div><br></div><div><div>Wed Nov 25 10:40:11 2015 myscript.prof</div><div><br></div><div> 7892315 function calls (7891054 primitive calls) in 7.940 seconds</div><div><br></div><div> Ordered by: internal time</div><div> List reduced from 1518 to 20 due to restriction <20></div><div><br></div><div> ncalls tottime percall cumtime percall filename:lineno(function)</div><div> 9086 2.693 0.000 6.706 0.001 inspect.py:247(getmembers)</div><div> 1952494 1.394 0.000 1.880 0.000 inspect.py:59(isclass)</div><div> 9092 1.030 0.000 1.030 0.000 {dir}</div><div> 1952642 0.600 0.000 0.600 0.000 {getattr}</div><div> 1972765 0.504 0.000 0.504 0.000 {isinstance}</div><div> 3 0.334 0.111 0.334 0.111 {method 'perform' of 'pycurl.Curl' objects}</div><div> 1883918 0.284 0.000 0.284 0.000 {method 'append' of 'list' objects}</div><div> 9087 0.221 0.000 0.221 0.000 {method 'sort' of 'list' objects}</div><div> 9051 0.172 0.000 6.911 0.001 reflectionhelper.py:51(isModuleMember)</div><div> 1 0.124 0.124 0.354 0.354 errors.py:17(<module>)</div><div> 1 0.088 0.088 0.230 0.230 params.py:8(<module>)</div><div> 8879 0.070 0.000 6.998 0.001 params.py:367(__setattr__)</div><div> 1 0.047 0.047 5.182 5.182 api.py:23(<module>)</div><div> 1 0.025 0.025 4.743 4.743 brokers.py:22(<module>)</div><div> 1 0.023 0.023 0.030 0.030 connectionspool.py:17(<module>)</div><div> 1 0.022 0.022 0.053 0.053 lxml.etree.pyx:1(PyMODINIT_FUNC PyInit_etree(void))</div><div> 118 0.019 0.000 4.684 0.040 params.py:45277(__init__)</div><div> 5 0.015 0.003 0.024 0.005 {built-in method strptime}</div><div> 1 0.012 0.012 0.013 0.013 socket.py:45(<module>)</div><div> 10 0.011 0.001 0.015 0.002 collections.py:288(namedtuple)</div></div><div><br></div><div>So it is not the classes, it is the code inspecting them on import.</div><div><br></div><div>Nir</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Nov 25, 2015 at 7:49 AM, John Hunter <span dir="ltr"><<a href="mailto:zhjwpku@gmail.com" target="_blank">zhjwpku@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Nir,<div><br></div><div>Attachment is my script and its profile.</div><div>Thanks a lot about your help!</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Nov 25, 2015 at 5:49 AM, Nir Soffer <span dir="ltr"><<a href="mailto:nsoffer@redhat.com" target="_blank">nsoffer@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><div>On Tue, Nov 24, 2015 at 3:49 PM, John Hunter <span dir="ltr"><<a href="mailto:zhjwpku@gmail.com" target="_blank">zhjwpku@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><div><div>On Tue, Nov 24, 2015 at 9:15 PM, Juan Hernández <span dir="ltr"><<a href="mailto:jhernand@redhat.com" target="_blank">jhernand@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span>On 11/24/2015 01:40 PM, John Hunter wrote:<br>
> Hi,<br>
><br>
> On Tue, Nov 24, 2015 at 5:18 PM, Oved Ourfali <<a href="mailto:oourfali@redhat.com" target="_blank">oourfali@redhat.com</a><br>
</span><span>> <mailto:<a href="mailto:oourfali@redhat.com" target="_blank">oourfali@redhat.com</a>>> wrote:<br>
><br>
> Hi<br>
><br>
> I discussed it with Juan (cc-ed).<br>
><br>
> There used to be a bug in the JDBC authenticion extension that<br>
> artificially delayed RESTAPI responses by 5 seconds:<br>
><br>
> brute force prevention login delay should not be applied to successful<br>
> login requests<br>
> <a href="https://bugzilla.redhat.com/1255814" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/1255814</a><br>
><br>
> That matches the description of the issue, but in theory it has been<br>
> fixed. I would suggest him to check that he is using the right version<br>
> of the extension.<br>
><br>
> I did not use the extension ovirt-engine-extension-aaa-jdbc, and I don't<br>
> think this bug matches my problem, because even there is only one line<br>
> in the python script, it still cost like 3 seconds, I don't think this is a<br>
> reasonable time as when I import other package, it cost almost no time.<br>
><br>
> Can you explain why this import line costs so much time?<br>
><br>
<br>
</span>If you are using 3.6 then you are using ovirt-engine-extension-aaa-jdbc,<br>
as it is enabled by default, but looks like it isn't related to your<br>
problem.<br>
<br>
That line takes a long time to execute because it has to process two<br>
large Python modules: the "params" module that contains a class per each<br>
type used by the API (393 classes) and the "brokers" module that<br>
contains a class per each resource used by the API (358 classes). That<br>
makes a total of 751 classes. In my environment it takes 0.9 seconds,<br>
approx. You may want to use the python profile in your environment and<br>
share the results:<br>
<br>
$ cat > profile.py <<.<br>
import cProfile<br>
cProfile.run("from ovirtsdk.api import API")<br>
.<br>
<br>
$ python profile.py<br>
<br>
I won't be surprised to see this taking those 3 seconds in a slower<br>
environment.<br>
<br>
But even if this takes those 3 seconds it shouldn't be a big problem,<br>
because you shouldn't be running that "from ... import ..." line<br>
frequently. Your program should do this once only, when it starts.<br>
<span><br></span></blockquote></div></div><div>Assume that I have two functions to implement, one is to list all the vms belong</div><div>to the user, and the other is to retrieve one vm's virt-viewer connection file, as </div><div>far as I can see, I have to write two python scripts and import the ovirtsdk.api in both</div><div>scripts, each script has to take the 3 seconds :(</div></div></div></div></blockquote><div><br></div></div></div><div>No, you have two functions, which can be in the same module, or in different modules, depending on how you want to organize your code.</div><div><br></div><div>Python imports are done only once, on the first time you import a module anywhere. The module is stored in sys.modules, and the next import fetch the module object from sys.modules.</div><div><br></div><div>Can you share a profile of a simple script importing the Python sdk?</div><div><br></div><div>Do:</div><div><br></div><div> python -m cProfile -o myscript.prof myscript.py</div><div><br></div><div>The profile will be stored in myscript.prof.</div><div><br></div><div>For viewing the profile, I recommend to use the profile-stats tool:</div><div><a href="https://github.com/oVirt/vdsm/blob/master/contrib/profile-stats" target="_blank">https://github.com/oVirt/vdsm/blob/master/contrib/profile-stats</a><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div><div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div>How can I run the "from ... import ..." just once ?</div><span><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span>
><br>
> In addition we also know that retrieving large lists of objects with the<br>
> SDK is slow:<br>
><br>
> [RFE][performance] - generate large scale list running to slow.<br>
> <a href="https://bugzilla.redhat.com/1221238" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/1221238</a><br>
><br>
> We don't have a solution for that yet.<br>
><br>
> CC-ing Juan in case you have additional questions.<br>
><br>
><br>
> On Mon, Nov 23, 2015 at 11:27 AM, John Hunter <<a href="mailto:zhjwpku@gmail.com" target="_blank">zhjwpku@gmail.com</a><br>
</span><span>> <mailto:<a href="mailto:zhjwpku@gmail.com" target="_blank">zhjwpku@gmail.com</a>>> wrote:<br>
><br>
> Hi guys,<br>
><br>
> I am using the ovirt-engine-sdk-python to communicate with the<br>
> ovirt-engine,<br>
> I am ok to list the vms but the processing time is too long,<br>
> like 4.5 seconds,<br>
> and this line:<br>
> from ovirtsdk.api import API<br>
> take almost 3 seconds.<br>
><br>
> This seems a little bit longer than I expected it to be, so I am<br>
> asking is there<br>
> a quicker way to communicate with the ovirt-engine?<br>
><br>
<br>
--<br>
</span>Dirección Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta<br>
3ºD, 28016 Madrid, Spain<br>
Inscrita en el Reg. Mercantil de Madrid – C.I.F. B82657941 - Red Hat S.L.<br>
</blockquote></span></div><br><br clear="all"><span><div><br></div>-- <br><div><div dir="ltr"><div><div dir="ltr"><div>Best regards<br></div><div>Junwang Zhao</div><div>Department of Computer Science &Technology</div><div>Peking University</div><div>Beijing, 100871, PRC</div></div></div></div></div>
</span></div></div>
<br></div></div><span>_______________________________________________<br>
Devel mailing list<br>
<a href="mailto:Devel@ovirt.org" target="_blank">Devel@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/devel" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/devel</a><br></span></blockquote></div><br></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div><div dir="ltr"><div><div dir="ltr"><div>Best regards<br></div><div>Junwang Zhao</div><div>Department of Computer Science &Technology</div><div>Peking University</div><div>Beijing, 100871, PRC</div></div></div></div></div>
</div>
</div></div></blockquote></div><br></div>