Il 30/09/2014 17:03, Dan Kenigsberg ha scritto:
On Tue, Sep 30, 2014 at 10:23:47AM +0000, Daniel Helgenberger wrote:
On 30.09.2014 11:57, Piotr Kliczewski wrote:
----- Original Message -----
From: "Daniel Helgenberger" <daniel.helgenberger@m-box.de> To: "Piotr Kliczewski" <pkliczew@redhat.com>, "Dan Kenigsberg" <danken@redhat.com> Cc: "Francesco Romani" <fromani@redhat.com>, users@ovirt.org Sent: Tuesday, September 30, 2014 11:50:28 AM Subject: Re: [ovirt-users]?3.4: VDSM Memory consumption
Hello Piotr,
On 30.09.2014 08:37, Piotr Kliczewski wrote:
----- Original Message -----
From: "Dan Kenigsberg" <danken@redhat.com> To: "Daniel Helgenberger" <daniel.helgenberger@m-box.de>, pkliczew@redhat.com Cc: "Francesco Romani" <fromani@redhat.com>, users@ovirt.org Sent: Tuesday, September 30, 2014 1:11:42 AM Subject: Re: [ovirt-users]?3.4: VDSM Memory consumption
On Mon, Sep 29, 2014 at 09:02:19PM +0000, Daniel Helgenberger wrote: > Hello Francesco, > > -- > Daniel Helgenberger > m box bewegtbild GmbH > > P: +49/30/2408781-22 > F: +49/30/2408781-10 > ACKERSTR. 19 > D-10115 BERLIN > www.m-box.de www.monkeymen.tv > >> On 29.09.2014, at 22:19, Francesco Romani <fromani@redhat.com> wrote: >> >> ----- Original Message ----- >>> From: "Daniel Helgenberger" <daniel.helgenberger@m-box.de> >>> To: "Francesco Romani" <fromani@redhat.com> >>> Cc: "Dan Kenigsberg" <danken@redhat.com>, users@ovirt.org >>> Sent: Monday, September 29, 2014 2:54:13 PM >>> Subject: Re: [ovirt-users] 3.4: VDSM Memory consumption >>> >>> Hello Francesco, >>> >>>> On 29.09.2014 13:55, Francesco Romani wrote: >>>> ----- Original Message ----- >>>>> From: "Daniel Helgenberger" <daniel.helgenberger@m-box.de> >>>>> To: "Dan Kenigsberg" <danken@redhat.com> >>>>> Cc: users@ovirt.org >>>>> Sent: Monday, September 29, 2014 12:25:22 PM >>>>> Subject: Re: [ovirt-users] 3.4: VDSM Memory consumption >>>>> >>>>> Dan, >>>>> >>>>> I just reply to the list since I do not want to clutter BZ: >>>>> >>>>> While migrating VMs is easy (and the sampling is already running), >>>>> can >>>>> someone tell me the correct polling port to block with iptables? >>>>> >>>>> Thanks, >>>> Hi Daniel, >>>> >>>> there is indeed a memory profiling patch under discussion: >>>> http://gerrit.ovirt.org/#/c/32019/ >>>> >>>> but for your case we'll need a backport to 3.4.x and clearer install >>>> instructions, >>>> which I'll prepare as soon as possible. >>> I updated the BZ (and are now blocking 54321/tcp on one of my hosts). >>> and verified it is not reachable. As general info: This system I am >>> using is my LAB / Test / eval setup for a final deployment for ovirt >>> (then 3.5) in production; so it will go away some time in the future (a >>> few weeks / months). If I am the only one experiencing this problem >>> then >>> you might be better of allocating resources elsewhere ;) >> Thanks for your understanding :) >> >> Unfortunately it is true that developer resources aren't so abundant, >> but it is also true that memleaks should never be discarded easily and >> without >> due investigation, considering the nature and the role of VDSM. >> >> So, I'm all in for further investigation regarding this issue. >> >>>> As for your question: if I understood correctly what you are asking >>>> (still catching up the thread), if you are trying to rule out the >>>> stats >>>> polling >>>> made by Engine to this bad leak, one simple way to test is just to >>>> shutdown >>>> Engine, >>>> and let VDSMs run unguarded on hypervisors. You'll be able to command >>>> these >>>> VDSMs using vdsClient or restarting Engine. >>> As I said in my BZ comment this is not an option right now, but if >>> understand the matter correctly IPTABLES reject should ultimately do >>> the >>> same? >> Definitely yes! Just do whatever it is more convenient for you. >> > As you might have already seen in the BZ comment the leak stopped after > blocking the port. Though this is clearly no permanent option - please > let > me know if I can be of any more assistance! The immediate suspect in this situation is M2Crypto. Could you verify that by re-opening the firewall and setting ssl=False in vdsm.conf?
You should disable ssl on Engine side and restart both Engine and Vdsm (too bad I do not recall how that's done on Engine: Piotr, can you help?).
In vdc_options table there is option EncryptHostCommunication.
Please confirm the following procedure is correct:
1. Change Postgres table value: # sudo -u postgres psql -U postgres engine -c "update vdc_options set option_value = 'false' where option_name = 'EncryptHostCommunication';" engine=# SELECT * from vdc_options where option_name='EncryptHostCommunication'; option_id | option_name | option_value | version -----------+--------------------------+--------------+--------- 335 | EncryptHostCommunication | false | general (1 row)
2. Restart engine 3. On the hosts; grep ssl /etc/vdsm/vdsm.conf #ssl = true ssl = false
4. restart VDSM
I assume I have to set 'ssl = false' this on on all hosts?
Please to set it to false and restart the engine.
I believe that you need to update a bit more on vdsm side. Please follow [1] section "Configure ovirt-engine and vdsm to work in non-secure mode"
There is wrong name of the option and it should be EncryptHostCommunication.
I forgot; I suppose hosted-engine-ha is out of order because of disabled ssl?
Indeed. And in hosted-engine, too, I need someone else's help (Sando?) to tell how to disable ssl.
in /etc/ovirt-hosted-engine: hosted-engine.conf just change: vdsm_use_ssl=true to vdsm_use_ssl=false
hosted-engine --connect-storage Connecting Storage Server Traceback (most recent call last): File "/usr/share/vdsm/vdsClient.py", line 2578, in <module> code, message = commands[command][0](commandArgs) File "/usr/share/vdsm/vdsClient.py", line 712, in connectStorageServer res = self.s.connectStorageServer(serverType, spUUID, conList) File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request verbose=self.__verbose File "/usr/lib64/python2.6/xmlrpclib.py", line 1235, in request self.send_content(h, request_body) File "/usr/lib64/python2.6/xmlrpclib.py", line 1349, in send_content connection.endheaders() File "/usr/lib64/python2.6/httplib.py", line 908, in endheaders self._send_output() File "/usr/lib64/python2.6/httplib.py", line 780, in _send_output self.send(msg) File "/usr/lib64/python2.6/httplib.py", line 739, in send self.connect() File "/usr/lib64/python2.6/site-packages/vdsm/SecureXMLRPCServer.py", line 195, in connect cert_reqs=self.cert_reqs) File "/usr/lib64/python2.6/ssl.py", line 342, in wrap_socket suppress_ragged_eofs=suppress_ragged_eofs) File "/usr/lib64/python2.6/ssl.py", line 120, in __init__ self.do_handshake() File "/usr/lib64/python2.6/ssl.py", line 279, in do_handshake self._sslobj.do_handshake() SSLError: [Errno 8] _ssl.c:492: EOF occurred in violation of protocol
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com