Using memory-profiler in a docker container

Question

RAM is ramping up in my docker container and I aim to investigate the cause. I was recommended memory profiler to help me with this.

I was told the steps were as follows:

• Install https://pypi.org/project/memory-profiler/ in the image
• manually execute the program in the container using mprof run -o memory-profile.dat python3 -m vdx
• mprof plot memory-profile.dat

Now, I'm having trouble realizing these steps, since I am new to docker, python and the memory profiler.

For the first step, I included RUN pip3 install memory-profiler in the Dockerfile below RUN pip3 install -r requirements.txt

Then, below where it says COPY src /vdx I've added RUN mprof run -o memory-profile.dat python3 -m vdp Now, when I build and run the container everything seems to work as usual...

Container is being built:

Step 7/12 : RUN pip3 install -r requirements.txt
 ---> Using cache
 ---> 11ad28b8793a
Step 8/12 : RUN pip3 install memory-profiler
 ---> Using cache
 ---> 58885abbcd8b
Step 9/12 : COPY src /vdx
 ---> Using cache
 ---> 15e884cb94c5
Step 10/12 : WORKDIR /
 ---> Using cache
 ---> 93cdfcee7f4a
Step 11/12 : RUN mprof run -o memory-profile.dat python3 -m vdx
 ---> Using cache
 ---> de7b15fd74ef
Step 12/12 : ENTRYPOINT ["python3", "-m", "vdx"]
 ---> Using cache
 ---> 53675841da99
Successfully built 53675841da99

and the process runs smoothly ...

but no file memory-profile.dat is being created.

What am I doing wrong?

Answer 1

Notice there are two different stages in the life-cycle of a container: building and running. The Dockerfile applies to the building stage, whereas the ENTRYPOINT action in the Dockerfile applies to the running stage.

This means that RUN mprof run -o memory-profile.dat python3 -m vdx in the Dockerfile is probably not what you want, as you want to investigate the issue when the container is running and not when it's building, don't you?

Basically what you are doing is, you are running your app with the memory profiler at build time, and then you set an entrypoint without the memory profiler for runtime, so when you run the container, you actually don't have the output of the profiler because it's not being run.

You need to run the profiler at runtime, moving the instruction to the ENTRYPOINT. ENTRYPOINT ["mprof", "run", "-o", "memory-profile.dat", "python3", "-m", "vdx"].