Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

diagnosing memory leak in python

I've been trying to debug a memory leak in the Coopr package using objgraph: https://gist.github.com/3855150

I have it pinned down to a _SetContainer object, but can't seem to figure out why that object is persisting in memory. Here's the result of objgraph.show_refs:

show_refs

How do I go about finding the circular reference and how can I get the the garbage collector to collect the _SetContainer instance?

I previously thought that the class itself might have a self-reference (the tuple just below the class on the right in the image above). But objgraph always shows inherited classes always as having a self-referencing tuple. You can see a very simple test case here.

like image 989
Adam Greenhall Avatar asked Oct 18 '12 18:10

Adam Greenhall


People also ask

How do you diagnose memory leaks?

To find a memory leak, look at how much RAM the system is using. The Resource Monitor in Windows can be used to accomplish this. In Windows 8.1 and Windows 10: To open the Run dialogue, press Windows+R, then type "resmon" and click OK.

What causes memory leaks in Python?

Since Python uses reference counting as one of its primary garbage collection algorithms, these leaking objects are usually caused by objects holding a reference to them longer than they should.


1 Answers

It's mostly guessing from the output of objgraph, but it seems that the instance is in a cycle and its class has a __del__. In this situation, the object is kept alive forever in CPython. Check it with:

import gc; gc.collect(); print gc.garbage

http://docs.python.org/library/gc.html#gc.garbage

like image 111
Armin Rigo Avatar answered Oct 01 '22 06:10

Armin Rigo