Re: [ovirt-users] ?3.4: VDSM Memory consumption

From: "Dan Kenigsberg" <danken(a)redhat.com&gt; To: "Daniel Helgenberger" <daniel.helgenberger(a)m-box.de&gt;, pkliczew(a)redhat.com Cc: "Francesco Romani" <fromani(a)redhat.com&gt;, users(a)ovirt.org Sent: Tuesday, September 30, 2014 1:11:42 AM Subject: Re: [ovirt-users]?3.4: VDSM Memory consumption On Mon, Sep 29, 2014 at 09:02:19PM +0000, Daniel Helgenberger wrote: > Hello Francesco, > > -- > Daniel Helgenberger > m box bewegtbild GmbH > > P: +49/30/2408781-22 > F: +49/30/2408781-10 > ACKERSTR. 19 > D-10115 BERLIN > www.m-box.de www.monkeymen.tv > > > On 29.09.2014, at 22:19, Francesco Romani <fromani(a)redhat.com&gt; wrote: > > > > ----- Original Message ----- > >> From: "Daniel Helgenberger" <daniel.helgenberger(a)m-box.de&gt; > >> To: "Francesco Romani" <fromani(a)redhat.com&gt; > >> Cc: "Dan Kenigsberg" <danken(a)redhat.com&gt;, users(a)ovirt.org > >> Sent: Monday, September 29, 2014 2:54:13 PM > >> Subject: Re: [ovirt-users] 3.4: VDSM Memory consumption > >> > >> Hello Francesco, > >> > >>> On 29.09.2014 13:55, Francesco Romani wrote: > >>> ----- Original Message ----- > >>>> From: "Daniel Helgenberger" <daniel.helgenberger(a)m-box.de&gt; > >>>> To: "Dan Kenigsberg" <danken(a)redhat.com&gt; > >>>> Cc: users(a)ovirt.org > >>>> Sent: Monday, September 29, 2014 12:25:22 PM > >>>> Subject: Re: [ovirt-users] 3.4: VDSM Memory consumption > >>>> > >>>> Dan, > >>>> > >>>> I just reply to the list since I do not want to clutter BZ: > >>>> > >>>> While migrating VMs is easy (and the sampling is already running), can > >>>> someone tell me the correct polling port to block with iptables? > >>>> > >>>> Thanks, > >>> Hi Daniel, > >>> > >>> there is indeed a memory profiling patch under discussion: > >>> http://gerrit.ovirt.org/#/c/32019/ > >>> > >>> but for your case we'll need a backport to 3.4.x and clearer install > >>> instructions, > >>> which I'll prepare as soon as possible. > >> I updated the BZ (and are now blocking 54321/tcp on one of my hosts). > >> and verified it is not reachable. As general info: This system I am > >> using is my LAB / Test / eval setup for a final deployment for ovirt > >> (then 3.5) in production; so it will go away some time in the future (a > >> few weeks / months). If I am the only one experiencing this problem then > >> you might be better of allocating resources elsewhere ;) > > > > Thanks for your understanding :) > > > > Unfortunately it is true that developer resources aren't so abundant, > > but it is also true that memleaks should never be discarded easily and > > without > > due investigation, considering the nature and the role of VDSM. > > > > So, I'm all in for further investigation regarding this issue. > > > >>> As for your question: if I understood correctly what you are asking > >>> (still catching up the thread), if you are trying to rule out the stats > >>> polling > >>> made by Engine to this bad leak, one simple way to test is just to > >>> shutdown > >>> Engine, > >>> and let VDSMs run unguarded on hypervisors. You'll be able to command > >>> these > >>> VDSMs using vdsClient or restarting Engine. > >> As I said in my BZ comment this is not an option right now, but if > >> understand the matter correctly IPTABLES reject should ultimately do the > >> same? > > > > Definitely yes! Just do whatever it is more convenient for you. > > > As you might have already seen in the BZ comment the leak stopped after > blocking the port. Though this is clearly no permanent option - please let > me know if I can be of any more assistance! The immediate suspect in this situation is M2Crypto. Could you verify that by re-opening the firewall and setting ssl=False in vdsm.conf? You should disable ssl on Engine side and restart both Engine and Vdsm (too bad I do not recall how that's done on Engine: Piotr, can you help?).