<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Wed, Aug 31, 2016 at 8:06 PM, Federico Alberto Sayd <span dir="ltr"><<a href="mailto:fsayd@uncu.edu.ar" target="_blank">fsayd@uncu.edu.ar</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Hello Nir:</p>
<p><br>
</p>
<p>I followed your instructions , added the config file, restarted
vdsm, and today I have the vdsm logs from a host:</p>
<p><a href="https://drive.google.com/file/d/0ByrwZ1AkYuyeR1hmRm90a1R6MEk/view?usp=sharing" target="_blank">https://drive.google.com/file/<wbr>d/<wbr>0ByrwZ1AkYuyeR1hmRm90a1R6MEk/<wbr>view?usp=sharing</a></p>
<p>Please tell me if you see anything related to the memory issue.</p></div></blockquote><div><br></div><div>This logs start when vdsm is using 567640 kB (554 MiB) - very unusual.</div><div><br></div><div>The memory usage grow by 18 MiB during one day. No garbage collection</div><div>issues. This smells like we keep some data forever for no reason.</div><div><br></div><div><div><div>$ grep rss= vdsm-leak.log | head -n 1<br></div><div>Thread-33::DEBUG::2016-08-30 12:01:43,845::health::122::health::(_check_resources) user=1.73%, sys=1.65%, rss=567640 kB (+44), threads=57</div></div><div><br></div><div> $ grep rss= vdsm-leak.log | tail -n 1</div><div>Thread-33::DEBUG::2016-08-31 13:00:36,913::health::122::health::(_check_resources) user=4.18%, sys=1.87%, rss=586584 kB (+0), threads=52</div></div><div><br></div><div>I would like to see the logs since vdsm was started - do you have them?</div><div><br></div><div>Also, can you describe the workload on this hypervisor?</div><div><br></div><div>- how many vms are running at the same time</div><div>- how many vms are started and stopped per hour</div><div>- using default vdsm.conf? if not, please attach your conf</div><div><br></div><div><div>Nir</div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF">
<p><br>
</p>
<p>Thanks<br>
</p>
<br>
Federico<br>
<br>
<div>El 30/08/16 a las 03:47, Nir Soffer
escribió:<br>
</div><div><div class="gmail-h5">
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Tue, Aug 30, 2016 at 1:30 AM,
Federico Alberto Sayd <span dir="ltr"><<a href="mailto:fsayd@uncu.edu.ar" target="_blank"></a><a href="mailto:fsayd@uncu.edu.ar" target="_blank">fsayd@uncu.edu.ar</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">I
have issues with my ovirt setup related to memory
consumption. After<br>
upgrading to 4.0 I noted a considerable grow in vdsm
memory consumption.<br>
I suspect that the grow is related to a memory leak.<br>
</blockquote>
<div><br>
</div>
<div>We need more details, see bellow...</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><br>
When I boot up the system and activate the host the memory
consumption<br>
is about 600MB. After 5 days running and host in
maintenance mode the<br>
memory consumption is about 1,4 GB.<br>
<br>
I need to put my hosts in maintenance and reboot to free
memory.<br>
</blockquote>
<div><br>
</div>
<div>You can restart vdsm (systemctl restart vdsmd) instead,
running vms</div>
<div>are not effected by this.</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><br>
Can anyone help me to debug this problem?<br>
</blockquote>
<div><br>
</div>
<div>We had a memory in vdsm-4.18.5, fixed in vdsm-4.18.11.
Since you</div>
<div>are running 4.18.11, there may be another leak.</div>
<div><br>
</div>
<div>Please enable health monitoring by creating</div>
<div>/etc/vdsm/vdsm.conf.d/50-<wbr>health.conf</div>
<div><br>
</div>
<div>[devel]</div>
<div>health_monitor_enable = true</div>
<div><br>
</div>
<div>And restart vdsm.</div>
<div><br>
</div>
<div>Please run with this setting for couple of hours, maybe
one day,</div>
<div>and then share the vdsm logs from this timeframe.</div>
<div><br>
</div>
<div>You may disable health monitoring by setting</div>
<div><br>
</div>
<div>
<div>[devel]</div>
</div>
<div>health_monitor_enable = false</div>
<div><br>
</div>
<div>Or by renaming or deleting this configuration file:</div>
<div><br>
</div>
<div>/etc/vdsm/vdsm.conf.d/50-<wbr>health.conf.disabled</div>
<div><br>
</div>
<div>Nir</div>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</div></div></div>
</blockquote></div><br></div></div>