Re: [ovirt-users] vdsm memory consumption (ovirt 4.0)
On Wed, Aug 31, 2016 at 8:06 PM, Federico Alberto Sayd <fsayd(a)uncu.edu.ar>
wrote:
Hello Nir:
I followed your instructions , added the config file, restarted vdsm, and
today I have the vdsm logs from a host:
https://drive.google.com/file/d/0ByrwZ1AkYuyeR1hmRm90a1R6MEk/
view?usp=sharing
Please tell me if you see anything related to the memory issue.
This logs start when vdsm is using 567640 kB (554 MiB) - very unusual.
The memory usage grow by 18 MiB during one day. No garbage collection
issues. This smells like we keep some data forever for no reason.
$ grep rss= vdsm-leak.log | head -n 1
Thread-33::DEBUG::2016-08-30
12:01:43,845::health::122::health::(_check_resources) user=1.73%,
sys=1.65%, rss=567640 kB (+44), threads=57
$ grep rss= vdsm-leak.log | tail -n 1
Thread-33::DEBUG::2016-08-31
13:00:36,913::health::122::health::(_check_resources) user=4.18%,
sys=1.87%, rss=586584 kB (+0), threads=52
I would like to see the logs since vdsm was started - do you have them?
Also, can you describe the workload on this hypervisor?
- how many vms are running at the same time
- how many vms are started and stopped per hour
- using default vdsm.conf? if not, please attach your conf
Nir
Thanks
Federico
El 30/08/16 a las 03:47, Nir Soffer escribió:
On Tue, Aug 30, 2016 at 1:30 AM, Federico Alberto Sayd <
<fsayd@uncu.edu.ar>fsayd(a)uncu.edu.ar> wrote:
> I have issues with my ovirt setup related to memory consumption. After
> upgrading to 4.0 I noted a considerable grow in vdsm memory consumption.
> I suspect that the grow is related to a memory leak.
>
We need more details, see bellow...
>
> When I boot up the system and activate the host the memory consumption
> is about 600MB. After 5 days running and host in maintenance mode the
> memory consumption is about 1,4 GB.
>
> I need to put my hosts in maintenance and reboot to free memory.
>
You can restart vdsm (systemctl restart vdsmd) instead, running vms
are not effected by this.
>
> Can anyone help me to debug this problem?
>
We had a memory in vdsm-4.18.5, fixed in vdsm-4.18.11. Since you
are running 4.18.11, there may be another leak.
Please enable health monitoring by creating
/etc/vdsm/vdsm.conf.d/50-health.conf
[devel]
health_monitor_enable = true
And restart vdsm.
Please run with this setting for couple of hours, maybe one day,
and then share the vdsm logs from this timeframe.
You may disable health monitoring by setting
[devel]
health_monitor_enable = false
Or by renaming or deleting this configuration file:
/etc/vdsm/vdsm.conf.d/50-health.conf.disabled
Nir
Attachments:
- attachment.html (text/html — 5.9 KB)