[ovirt-users] vdsm memory consumption (ovirt 4.0)

[Nir Soffer]

On Wed, Aug 31, 2016 at 8:06 PM, Federico Alberto Sayd <fsayd at uncu.edu.ar>
wrote:

> Hello Nir:
>
>
> I followed your instructions , added the config file, restarted vdsm, and
> today I have the vdsm logs from a host:
>
> https://drive.google.com/file/d/0ByrwZ1AkYuyeR1hmRm90a1R6MEk/
> view?usp=sharing
>
> Please tell me if you see anything related to the memory issue.
>

This logs start when vdsm is using 567640 kB (554 MiB) - very unusual.

The memory usage grow by 18 MiB during one day. No garbage collection
issues. This smells like we keep some data forever for no reason.

$ grep rss= vdsm-leak.log | head -n 1
Thread-33::DEBUG::2016-08-30
12:01:43,845::health::122::health::(_check_resources) user=1.73%,
sys=1.65%, rss=567640 kB (+44), threads=57

 $ grep rss= vdsm-leak.log | tail -n 1
Thread-33::DEBUG::2016-08-31
13:00:36,913::health::122::health::(_check_resources) user=4.18%,
sys=1.87%, rss=586584 kB (+0), threads=52

I would like to see the logs since vdsm was started  - do you have them?

Also, can you describe the workload on this hypervisor?

- how many vms are running at the same time
- how many vms are started and stopped per hour
- using default vdsm.conf? if not, please attach your conf

Nir

>
> Thanks
>
> Federico
>
> El 30/08/16 a las 03:47, Nir Soffer escribió:
>
> On Tue, Aug 30, 2016 at 1:30 AM, Federico Alberto Sayd <
> <fsayd at uncu.edu.ar>fsayd at uncu.edu.ar> wrote:
>
>> I have issues with my ovirt setup related to memory consumption. After
>> upgrading to 4.0 I noted a considerable grow in vdsm memory consumption.
>> I suspect that the grow is related to a memory leak.
>>
>
> We need more details, see bellow...
>
>
>>
>> When I boot up the system and activate the host the memory consumption
>> is about 600MB. After 5 days running and host in maintenance mode the
>> memory consumption is about 1,4 GB.
>>
>> I need to put my hosts in maintenance and reboot to free memory.
>>
>
> You can restart vdsm (systemctl restart vdsmd) instead, running vms
> are not effected by this.
>
>
>>
>> Can anyone help me to debug this problem?
>>
>
> We had a memory in vdsm-4.18.5, fixed  in vdsm-4.18.11. Since you
> are running 4.18.11, there may be another leak.
>
> Please enable health monitoring by creating
> /etc/vdsm/vdsm.conf.d/50-health.conf
>
> [devel]
> health_monitor_enable = true
>
> And restart vdsm.
>
> Please run with this setting for couple of hours, maybe one day,
> and then share the vdsm logs from this timeframe.
>
> You may disable health monitoring by setting
>
> [devel]
> health_monitor_enable = false
>
> Or by renaming or deleting this configuration file:
>
> /etc/vdsm/vdsm.conf.d/50-health.conf.disabled
>
> Nir
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20160831/27719907/attachment-0001.html>