Hello! After upgrade from 3.4.4. to 3.5.0 I can see n all my 3 nodes PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND nod1 SPM running 0 VM 753 root 15 -5 17.208g 7.737g 10832 S 0.0 49.4 1:28.46 supervdsmServer nod2 running 3 VM 641 root 15 -5 17.573g 7.888g 10768 S 0.0 33.5 1:17.09 supervdsmServer nod3 running 2 VM 6391 root 15 -5 19.072g 8.646g 10844 S 9.3 44.1 38:17.05 supervdsmServer So Supervdsm server ocupy around 33-49% of memory alone! Also I've got *systemctl status supervdsmd* supervdsmd.service - "Auxiliary vdsm service for running helper functions as root" Loaded: loaded (/usr/lib/systemd/system/supervdsmd.service; static) Active: active (running) since Tue 2014-10-21 11:32:40 EEST; 23h ago Main PID: 753 (supervdsmServer) CGroup: name=systemd:/system/supervdsmd.service ââ753 /usr/bin/python /usr/share/vdsm/supervdsmServer --sockfile /var/run/vdsm/svdsm.sock Oct 21 11:39:51 nod1 daemonAdapter[753]: Process Process-4: Oct 21 11:39:51 nod1 daemonAdapter[753]: Traceback (most recent call last): Oct 21 11:39:51 nod1 daemonAdapter[753]: File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap Oct 21 11:39:51 nod1 daemonAdapter[753]: self.run() Oct 21 11:39:51 nod1 daemonAdapter[753]: File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run Oct 21 11:39:51 nod1 daemonAdapter[753]: self._target(*self._args, **self._kwargs) Oct 21 11:39:51 nod1 daemonAdapter[753]: File "/usr/share/vdsm/supervdsmServer", line 242, in child Oct 21 11:39:51 nod1 daemonAdapter[753]: pipe.recv() Oct 21 11:39:51 nod1 daemonAdapter[753]: IOError: [Errno 4] Interrupted system call Any ideeas, hints?
Might hit https://bugzilla.redhat.com/show_bug.cgi?id=1142647 On Wed, Oct 22, 2014 at 10:45 AM, Gabi C <gabicr@gmail.com> wrote:
Hello!
After upgrade from 3.4.4. to 3.5.0 I can see n all my 3 nodes
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
nod1 SPM running 0 VM 753 root 15 -5 17.208g 7.737g 10832 S 0.0 49.4 1:28.46 supervdsmServer
nod2 running 3 VM 641 root 15 -5 17.573g 7.888g 10768 S 0.0 33.5 1:17.09 supervdsmServer
nod3 running 2 VM 6391 root 15 -5 19.072g 8.646g 10844 S 9.3 44.1 38:17.05 supervdsmServer
So Supervdsm server ocupy around 33-49% of memory alone!
Also I've got *systemctl status supervdsmd*
supervdsmd.service - "Auxiliary vdsm service for running helper functions as root" Loaded: loaded (/usr/lib/systemd/system/supervdsmd.service; static) Active: active (running) since Tue 2014-10-21 11:32:40 EEST; 23h ago Main PID: 753 (supervdsmServer) CGroup: name=systemd:/system/supervdsmd.service ââ753 /usr/bin/python /usr/share/vdsm/supervdsmServer --sockfile /var/run/vdsm/svdsm.sock
Oct 21 11:39:51 nod1 daemonAdapter[753]: Process Process-4: Oct 21 11:39:51 nod1 daemonAdapter[753]: Traceback (most recent call last): Oct 21 11:39:51 nod1 daemonAdapter[753]: File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap Oct 21 11:39:51 nod1 daemonAdapter[753]: self.run() Oct 21 11:39:51 nod1 daemonAdapter[753]: File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run Oct 21 11:39:51 nod1 daemonAdapter[753]: self._target(*self._args, **self._kwargs) Oct 21 11:39:51 nod1 daemonAdapter[753]: File "/usr/share/vdsm/supervdsmServer", line 242, in child Oct 21 11:39:51 nod1 daemonAdapter[753]: pipe.recv() Oct 21 11:39:51 nod1 daemonAdapter[753]: IOError: [Errno 4] Interrupted system call
Any ideeas, hints?
Another note: After rebooting all 3 nodes and starting some VMs everything seem fine!!! Also systemctl status supervdsmd supervdsmd.service - "Auxiliary vdsm service for running helper functions as root" Loaded: loaded (/usr/lib/systemd/system/supervdsmd.service; static) Active: active (running) since Wed 2014-10-22 11:48:34 EEST; 4min 34s ago Main PID: 621 (supervdsmServer) CGroup: name=systemd:/system/supervdsmd.service ââ621 /usr/bin/python /usr/share/vdsm/supervdsmServer --sockfile /var/run/vdsm/svdsm.sock NO MORE ERRORS!! ...see my first email.... On Wed, Oct 22, 2014 at 11:13 AM, Gabi C <gabicr@gmail.com> wrote:
Might hit https://bugzilla.redhat.com/show_bug.cgi?id=1142647
On Wed, Oct 22, 2014 at 10:45 AM, Gabi C <gabicr@gmail.com> wrote:
Hello!
After upgrade from 3.4.4. to 3.5.0 I can see n all my 3 nodes
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
nod1 SPM running 0 VM 753 root 15 -5 17.208g 7.737g 10832 S 0.0 49.4 1:28.46 supervdsmServer
nod2 running 3 VM 641 root 15 -5 17.573g 7.888g 10768 S 0.0 33.5 1:17.09 supervdsmServer
nod3 running 2 VM 6391 root 15 -5 19.072g 8.646g 10844 S 9.3 44.1 38:17.05 supervdsmServer
So Supervdsm server ocupy around 33-49% of memory alone!
Also I've got *systemctl status supervdsmd*
supervdsmd.service - "Auxiliary vdsm service for running helper functions as root" Loaded: loaded (/usr/lib/systemd/system/supervdsmd.service; static) Active: active (running) since Tue 2014-10-21 11:32:40 EEST; 23h ago Main PID: 753 (supervdsmServer) CGroup: name=systemd:/system/supervdsmd.service ââ753 /usr/bin/python /usr/share/vdsm/supervdsmServer --sockfile /var/run/vdsm/svdsm.sock
Oct 21 11:39:51 nod1 daemonAdapter[753]: Process Process-4: Oct 21 11:39:51 nod1 daemonAdapter[753]: Traceback (most recent call last): Oct 21 11:39:51 nod1 daemonAdapter[753]: File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap Oct 21 11:39:51 nod1 daemonAdapter[753]: self.run() Oct 21 11:39:51 nod1 daemonAdapter[753]: File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run Oct 21 11:39:51 nod1 daemonAdapter[753]: self._target(*self._args, **self._kwargs) Oct 21 11:39:51 nod1 daemonAdapter[753]: File "/usr/share/vdsm/supervdsmServer", line 242, in child Oct 21 11:39:51 nod1 daemonAdapter[753]: pipe.recv() Oct 21 11:39:51 nod1 daemonAdapter[753]: IOError: [Errno 4] Interrupted system call
Any ideeas, hints?
On Wed, Oct 22, 2014 at 11:13:27AM +0300, Gabi C wrote:
Might hit https://bugzilla.redhat.com/show_bug.cgi?id=1142647
Yeah, it sounds like the same issue, which is to be fixed (as much as oVirt matters) in ovirt-3.5.1. The underlying libgfapi https://bugzilla.redhat.com/show_bug.cgi?id=1093594 may take longer to solve.
Do you thnik that installing http://resources.ovirt.org/pub/ovirt-3.5-snapshot/rpm/fc19/x86_64/vdsm-4.16.... will solve that? On Wed, Oct 22, 2014 at 12:41 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Wed, Oct 22, 2014 at 11:13:27AM +0300, Gabi C wrote:
Might hit https://bugzilla.redhat.com/show_bug.cgi?id=1142647
Yeah, it sounds like the same issue, which is to be fixed (as much as oVirt matters) in ovirt-3.5.1. The underlying libgfapi https://bugzilla.redhat.com/show_bug.cgi?id=1093594 may take longer to solve.
On Thu, Oct 23, 2014 at 10:19:59AM +0300, Gabi C wrote:
Do you thnik that installing http://resources.ovirt.org/pub/ovirt-3.5-snapshot/rpm/fc19/x86_64/vdsm-4.16.... will solve that?
Let's look up the git hash: http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=shortlog;h=3733cdf has "gluster: Temporary fix for supervdsm memory leak" in it. So the answer is yes.
Thanks! I guess that enabling ovirt-nightly repos should be enough in order to install with all the dependencies, right? On Thu, Oct 23, 2014 at 11:30 AM, Dan Kenigsberg <danken@redhat.com> wrote:
On Thu, Oct 23, 2014 at 10:19:59AM +0300, Gabi C wrote:
Do you thnik that installing
http://resources.ovirt.org/pub/ovirt-3.5-snapshot/rpm/fc19/x86_64/vdsm-4.16....
will solve that?
Let's look up the git hash: http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=shortlog;h=3733cdf has "gluster: Temporary fix for supervdsm memory leak" in it.
So the answer is yes.
FYI Enabled nightly and updated vdsm* + mom.I'll let you know results, most likely tomorrow morning..... On Thu, Oct 23, 2014 at 12:53 PM, Gabi C <gabicr@gmail.com> wrote:
Thanks!
I guess that enabling ovirt-nightly repos should be enough in order to install with all the dependencies, right?
On Thu, Oct 23, 2014 at 11:30 AM, Dan Kenigsberg <danken@redhat.com> wrote:
On Thu, Oct 23, 2014 at 10:19:59AM +0300, Gabi C wrote:
Do you thnik that installing
http://resources.ovirt.org/pub/ovirt-3.5-snapshot/rpm/fc19/x86_64/vdsm-4.16....
will solve that?
Let's look up the git hash: http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=shortlog;h=3733cdf has "gluster: Temporary fix for supervdsm memory leak" in it.
So the answer is yes.
Just to let you know: all good! ..no more memory leak On Thu, Oct 23, 2014 at 3:48 PM, Gabi C <gabicr@gmail.com> wrote:
FYI Enabled nightly and updated vdsm* + mom.I'll let you know results, most likely tomorrow morning.....
On Thu, Oct 23, 2014 at 12:53 PM, Gabi C <gabicr@gmail.com> wrote:
Thanks!
I guess that enabling ovirt-nightly repos should be enough in order to install with all the dependencies, right?
On Thu, Oct 23, 2014 at 11:30 AM, Dan Kenigsberg <danken@redhat.com> wrote:
On Thu, Oct 23, 2014 at 10:19:59AM +0300, Gabi C wrote:
Do you thnik that installing
http://resources.ovirt.org/pub/ovirt-3.5-snapshot/rpm/fc19/x86_64/vdsm-4.16....
will solve that?
Let's look up the git hash: http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=shortlog;h=3733cdf has "gluster: Temporary fix for supervdsm memory leak" in it.
So the answer is yes.
participants (2)
-
Dan Kenigsberg -
Gabi C