On Tue, Dec 7, 2021 at 6:12 PM Chris Adams <cma@cmadams.net> wrote:
> Top differences
> /usr/lib64/python3.6/site-packages/libvirt.py:442: size=295 MiB (+285 MiB), count=5511282 (+5312311), average=56 B
> /usr/lib64/python3.6/json/decoder.py:355: size=73.9 MiB (+70.2 MiB), count=736108 (+697450), average=105 B
> /usr/lib64/python3.6/logging/__init__.py:1630: size=44.2 MiB (+43.8 MiB), count=345704 (+342481), average=134 B
> /usr/lib64/python3.6/site-packages/libvirt.py:5695: size=30.3 MiB (+30.0 MiB), count=190449 (+188665), average=167 B
> /usr/lib/python3.6/site-packages/vdsm/host/stats.py:138: size=12.1 MiB (+11.4 MiB), count=75366 (+70991), average=168 B
> /usr/lib/python3.6/site-packages/vdsm/utils.py:358: size=10.4 MiB (+9968 KiB), count=70204 (+65272), average=156 B

That's quite significant!

> Top block
> 5511282 memory blocks: 302589.8 KiB
>   File "/usr/lib64/python3.6/site-packages/libvirt.py", line 442
>     ret = libvirtmod.virEventRunDefaultImpl()
>   File "/usr/lib/python3.6/site-packages/vdsm/common/libvirtconnection.py", line 69
>     libvirt.virEventRunDefaultImpl()
>   File "/usr/lib/python3.6/site-packages/vdsm/common/concurrent.py", line 260
>     ret = func(*args, **kwargs)

You should check where these "ret" objects (of libvirt.py:442) are
stored: 5,511,282 is a lot of small objects (average: 56 bytes)! Maybe
they are stored in a list and never destroyed.

Maybe it's a reference leak in the libvirtmod.virEventRunDefaultImpl()
function of "libvirtmod" C extension: missing Py_DECREF() somewhere.

Or something somehow prevents to delete these projects object. For
example, an exception is stored somewhere which keeps all variables
alive (in Python 3, an exception stores a traceback object which keeps
all variables of all frames alive).

On GitHub and GitLab, I found the following code. Maybe there are
minor differences in the versions that you are using.

https://gitlab.com/libvirt/libvirt-python
(I built the code locally to get build/libvirt.py)

build/libvirt.c:
---
PyObject *
libvirt_intWrap(int val)
{
    return PyLong_FromLong((long) val);
}

PyObject *
libvirt_virEventRunDefaultImpl(PyObject *self ATTRIBUTE_UNUSED,
PyObject *args ATTRIBUTE_UNUSED) {
    PyObject *py_retval;
    int c_retval;
    LIBVIRT_BEGIN_ALLOW_THREADS;
    c_retval = virEventRunDefaultImpl();
    LIBVIRT_END_ALLOW_THREADS;
    py_retval = libvirt_intWrap((int) c_retval);
    return py_retval;
}

static PyMethodDef libvirtMethods[] = {
    { (char *)"virEventRunDefaultImpl",
libvirt_virEventRunDefaultImpl, METH_VARARGS, NULL },
    ...
    {NULL, NULL, 0, NULL}
};
---

This code looks correct and straightforward. Is it possible that
internally virEventRunDefaultImpl() calls a Python memory allocator?

build/libvirt.py:
---
def virEventRunDefaultImpl():
    ret = libvirtmod.virEventRunDefaultImpl()
    if ret == -1:
        raise libvirtError('virEventRunDefaultImpl() failed')
    return ret
---

Again, this code looks correct and straightforward.

https://github.com/oVirt/vdsm/blob/37ed5c279c2dd9c9bb06329d674882e0f98f34d6/lib/vdsm/common/libvirtconnection.py

vdsm/common/libvirtconnection.py:
---
    def __run(self):
        try:
            libvirt.virEventRegisterDefaultImpl()
            while self.run:
                libvirt.virEventRunDefaultImpl()
        finally:
            self.run = False
---

libvirt.virEventRunDefaultImpl() result is ignored and so I don't see
anything obvious which would explain a leak.


Sometimes, looking at the top function is misleading since the
explanation can be found in one of the caller functions.

For example, which function creates 70.2 MiB of objects from a JSON
document? What calls json/decoder.py:355?

Victor
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/O5OAA6KNLINLRT2VYKNBI2PPH6UIYR4A/