Django project eats memory

Question

I have a django project, and a problem - it eats a lot of memory and loads hosting too much.

How can I find the problem places in the project which eat a lot of memory?

Answer 1

If you're using Django with DEBUG = True then Django logs every database query which can quickly mount up and use a substantial amount of memory.

If you're not running in DEBUG mode, then take a look at gc module and in particular try adding gc.set_debug(gc.DEBUG_LEAK) to settings.py. This will show you a great deal of information about what objects are using memory.

Answer 2

In general for debugging/profiling, I suggest django-debug-toolbar as a starting location as well as the various tips in:

http://docs.djangoproject.com/en/dev/topics/db/optimization/

However this won't give memory usage info. If you really need that, you can try some middleware using pympler to log memory usage while debugging and run the development server.

http://www.rkblog.rk.edu.pl/w/p/profiling-django-object-size-and-memory-usage-pympler/?c=1

I've found that doing this grinds my webapps to a near-halt and then there are the problems from using the dev-webserver (e.g., media files not getting served).

But as others said your best bet is to set DEBUG=False:

http://docs.djangoproject.com/en/dev/faq/models/#why-is-django-leaking-memory