Python memory profiler plotting

Question

I am trying to log my memory consumption per function. I am using the memory_profiler package, and I also get a final graph with my total memory consumption, but not per function as described in this nice tutorial

I am using the following script test.py

import time
from memory_profiler import profile

#-------------------------------
@profile
def test1():
    a = [1] * (10 ** 6)
    b = [2] * (2 * 10 ** 7)
    del b
    return a

#-------------------------------
@profile
def test2():
    a = [1] * (10 ** 6)
    b = [2] * (2 * 10 ** 7)
    del b
    return a

#-------------------------------
def main():

    l1 = []
    l2 = []

    for i in range (5) :
        print "Iteration", i
        l1.append(test1())
        l2.append(test2())
        time.sleep(1)
        if i == 2 :
            del l1
            l1 = []

#-------------------------------
if __name__ == "__main__":
    main()

Running it via

mprof run test.py

I get some nice output like

Line #    Mem usage    Increment   Line Contents
================================================
11     46.2 MiB      0.0 MiB   @profile
12                             def test2():
13     53.8 MiB      7.6 MiB       a = [1] * (10 ** 6)
14    206.7 MiB    152.9 MiB       b = [2] * (2 * 10 ** 7)
15     53.8 MiB   -152.9 MiB       del b
16     53.8 MiB      0.0 MiB       return a

But the plotting it via

mprof plot

only shows the total memory consumption, and not these nice brakets, showing the execution of the different functions as described in the link. Does anyone know how to enable this?

Answer 1

I am profiling a Django app. Here's how I got it to work.

As @Lukic indicated, remove from memory_profiler import profile from your file. Yes, you will get an error in your editor. Yes, you will get an error if you run this as a normal python file. However, if you run it with the mprof command it will succeed. Run Django using the following command:

mprof run --multiprocess --python python manage.py runserver --noreload

--multiprocess is important as more recent versions of Django may (I think) launch multiple processes even on a development server.

--noreload is important, otherwise Django will throw an error as it first launches the reloader process and then that launches that actual instance (which chokes on the @process decoroator)

In a separate command window I'm able to run mprof plot and see the output. However, it is not annotated with any functions. This is because the annotations are not written to the mprof output file until the Django server is stopped. Once the server is stopped I can run mprof plot and see the overall memory utilization and the function run times.

Hopefully this helps - I just came across this question as I am starting to use mprof.

Environment: Windows 10, Python 3.5, Django 1.11

Reference: https://pypi.org/project/memory-profiler/ (There is a easily missed note about removing the import statement and it is not well explained)

Answer 2

You should remove the line:

from memory_profiler import profile