On Wed, Apr 26, 2017 at 1:28 PM, Simone Tiraboschi <stirabos(a)redhat.com>
wrote:
On Wed, Apr 26, 2017 at 12:52 PM, Nir Soffer <nsoffer(a)redhat.com> wrote:
> On Wed, Apr 26, 2017 at 11:36 AM Gianluca Cecchi <
> gianluca.cecchi(a)gmail.com> wrote:
>
>> On Tue, Apr 25, 2017 at 11:28 PM, Nir Soffer <nsoffer(a)redhat.com> wrote:
>>
>>>
>>>
>>> Hi Gianluca,
>>>
>>> You can run this on the host:
>>>
>>> $ python -c "import yaml; print 'CLoader:', hasattr(yaml,
'CLoader')"
>>> CLoader: True
>>>
>>> If you get "CLoader: False", you have some packaging issue,
CLoader
>>> is available on all supported platforms.
>>>
>>> Nir
>>>
>>>
>>>> Thanks,
>>>> Gianluca
>>>>
>>>>
>>
>> It seems ok.
>>
>> [root@ovirt01 ovirt-hosted-engine-ha]# python -c "import yaml; print
>> 'CLoader:', hasattr(yaml, 'CLoader')"
>> CLoader: True
>> [root@ovirt01 ovirt-hosted-engine-ha]#
>>
>> Anyway see here a sample of the spikes that it cntinues to have.. from
>> 15% to 55% many times
>>
https://drive.google.com/file/d/0BwoPbcrMv8mvMy1xVUE3YzI2YVE
>> /view?usp=sharing
>>
>
> There are two issues in this video:
> - Memory leak, ovirt-ha-agent is using 1g of memory. It is very unlikely
> that it needs so much memory.
> - Unusual cpu usage - but not the kind of usage related to yaml parsing.
>
> I would open two bugs for this. We have seen the first issue few month
> ago, and
> we did nothing about it so the memory leak was not fixed.
>
> To understand the unusual cpu usage, we need to integrate yappi into
> ovirt-ha-agent,
> and do some profiling to understand where cpu time is spent.
>
> Simone, can you do something based on these patches?
>
https://gerrit.ovirt.org/#/q/topic:generic-profiler
>
> I hope to get these patches merged soon.
>
>
Absolutely at this point.
On 4.1.1, the 96% of the cpu time of ovirt-ha-agent is still spent in
connect() in /usr/lib/python2.7/site-packages/vdsm/jsonrpcvdscli.py
and the 95.98% is in Schema.__init__
in /usr/lib/python2.7/site-packages/vdsm/api/vdsmapi.py
So it's still the parsing of the api yaml schema.
On master we already merged a patch to reuse an existing connection if
available and this should mitigate/resolve the issue:
https://gerrit.ovirt.org/73757/
It's still not that clear why we are facing this kind of performance
regression.
Nir
>
>
>>
>>
>> The host is an Intel NUC6i5 with 32Gb of ram. There are the engine, an
>> F25 guest and a C7 desktop VMs running, without doing almost anything.
>>
>> Gianluca
>>
>>