How to connect Emacs' Elpy in-buffer python interpreter to docker container?

Question

I am getting started working on a Django app that will run within a Docker container. I would like to be able to use interactive python from within Emacs which executes the code within the context of the Docker container and the Django app. Essentially I want the Python interpreter that is used by Emacs to be running within the Docker container.

I have managed to achieve this for the most part by creating a modified Dockerfile which is only different from my actual Dockerfile container in that instead of calling a Django command it ends with:

CMD ./manage.py shell

This means that when I run that Docker container I get a Django interpreter that is actually running within the container. I have Elpy invoke this when starting a python shell by adding:

(setq python-shell-interpreter "/path_to_my_script/run_shell.sh")

to my .emacs.d/init.el file. Where the "/path_to_my_script/run_shell.sh" script is a script which runs the container.

This is actually working for the most part. I am able to highlight a line of code such as

from django.utils import timezone

and run the Emacs command elpy-shell-send-region-or-buffer and it will execute the code. I can then highlight

print(timezone.now())

and it will print the current time. I have verified that this code is in fact running on my container by running:

import os
print(os.getcwd())

and seeing the home directory for my Docker container's user.

The issue I am still having with this setup is when I try to execute multiple lines of python at the same time using the same elpy-shell-send-region-or-buffer command. When I highlight more than one line and execute that command I get the following error within the python shell:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/local/lib/python3.5/codecs.py", line 895, in open
    file = builtins.open(filename, mode, buffering)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/py10324GOe'

I have been unable to find any answers to the following questions:

• Why is codecs.py trying to read a tmp file?
• What are all the temporary files created by Python for? (If you have some python code running and look in your /tmp directory you'll probably see lots of file like py271761jE.)
• Why is the /tmp directory empty in my Docker container? It seems like Python is wanting to create and read from files in that directory but is failing.
• How can I debug this?

If anyone has any answers to any of these questions it would be a huge help. I have been unable to turn up any information about this issue. Googling about py* files in the /tmp directory just turns up the tempfile package.

I don't know what's going on but my current theory is that when a region with multiple lines is sent to the python shell it needs to deal with newline characters so it tries to use the codecs file. I think somehow this file relies on temporary files and the docker container is messing that up. I don't know how close this theory is but I don't know how to investigate it further.

Answer 1

Why is codecs.py trying to read a tmp file?

I'm pretty sure this is what's happening:

1. Emacs writes code to /tmp/py10324GOe
2. Emacs instructs Python to load /tmp/py10324GOe
3. Python fails because the container cannot access the host's /tmp/. The container has its own, separate /tmp/ directory.

The simplest way to fix this would be to link the host's /tmp/ directory to the container's /tmp/ directory. Some googling leads me to a command like:

docker run ... -v /tmp:/tmp

I haven't tested that code, but it seems like the right direction.