Re: [ovirt-devel] [VDSM] health: Introduce Vdsm health monitoring
----- Original Message -----
From: "Nir Soffer" <nsoffer(a)redhat.com>
To: "devel" <devel(a)ovirt.org>, "Francesco Romani"
<fromani(a)redhat.com>
Sent: Friday, January 15, 2016 9:52:13 AM
Subject: Re: [VDSM] health: Introduce Vdsm health monitoring
I tested with https://gerrit.ovirt.org/#/c/51630
libvirt.virDomain object does not leak any more, but pthreading object
are still leaking,
and the number of uncollectible objects is still very high (12,915).
Each time we start and stop a vm, we leak about 6000 objects which are part
of a reference cycle, and each time we create and leak 27 pthreading
locks/conditions
which are probably the reason for this leak (because they implement
__del__ or reference object
implementing __del__.
*Surely* this deserve more investigation, *most likley* there is more to be fixed,
and *surely* I'll invest more energy in research, but I think we should also apply
some grains of salt to the raw results gc gives us.
I've been running a test overnight like
- start 20 vms
- sleep 2m
- stop 20 vms
- sleep 30s
repeated 100 times, monitoring VmRSS (/proc/$VDSM_PID/status) and I didn't observed
any _significant_ leak.
The VmRSS growth was pretty low, almost flat. So something doesn't click here.
I mean, 6000 objects (at least) per vm each time *should* be noticeable after all!
Once 51630 is in good shape I'll broad the net and go hunt for more virt leaks.
Bests,
--
Francesco Romani
RedHat Engineering Virtualization R & D
Phone: 8261328
IRC: fromani