Pycharm use Docker Container Python as Remote Interpreter

Question

I am trying to use the python in a docker container on a remote machine as the interpreter in Pycharm. Since that is a mouthful, here is a diagram:

There is a Jupyter Notebook running in the container, which I am able to connect to through my local browser (although this is just for testing the connection). The command I am using to launch the Docker container is

docker run --runtime=nvidia -it --rm --shm-size=2g -v /home/timo/storage:/storage -v /etc/passwd:/etc/passwd -v /etc/group:/etc/group --ulimit memlock=-1 -p 8888:8888 -p 7722:22 --ipc=host latest:latest

I can forward the port 8888 which the Jupyter notebook is running on with ssh -L 8888:0.0.0.0:8888 BBB.BBB.BBB.BBB and thus use it on the local machine. But I don't much like using Jupyter for developing and would like to use the Python interpreter in the Docker Container in Pycharm.

When I select "Add Python Interpreter" in Pycharm, I get the following options:

The documentation for Pycharm suggests using the "Add Python Interpreter/Docker" tool which looks like this:

However the documentation doesn't say how to set up the Docker container and the connections if the Docker is on a remote machine.

So my questions are: should I use a Unix or a TCP socket to connect to my remote docker? Or should I somehow forward all the relevant ports from the container and use the "SSH Interpreter" option? And if so, how do I set this all up? Am I setting up my Docker Container properly in the first place?

I think I have trawled through every forum and online resource, over the last two days, but have not come any closer to getting this to work. I have also tried to get this to work in Spyder, but to no avail either. So any advice is very appreciated!

Many thanks!

Answer 1

Thank you for depicting the dilemma so poignantly and clearly in your cartoon :-). My colleague and I were trying to do something similar and what ultimately worked beautifully was creating an SSH config directly to the Docker container jumping from the remote machine, and then setting it as a remote SSH interpreter so that pycharm doesn't even realize it's a Docker container. It also works well for vscode.

1. set up ssh service in docker container (subset of steps in https://dev.to/s1ntaxe770r/how-to-setup-ssh-within-a-docker-container-i5i, port22 stuff wasn't needed)
   • docker exec -it <container> bash: create admin interactive prompt for docker
   • apt-get install openssh-server
   • service ssh start
   • confirm with service ssh status -> * sshd is running
2. determine IP and test SSHing from remote machine into container (adapted from https://phoenixnap.com/kb/how-to-ssh-into-docker-container, steps 2 and 3)
   • from normal command prompt on remote machine (not within container): docker inspect -f "{{ .NetworkSettings.IPAddress }}" <container> to get container IP
   • test: ping -c 3 <container_ip>
   • ssh: ssh <container_ip>; should drop you into the container as your user; however, requires container to be configured properly (docker run cmd has -v /etc/passwd:/etc/passwd:ro \ etc.). It may ask for a password. note: if you do this for a different container later that is assigned the same IP, you will get a warning and may need to delete the previous key from known_hosts; just follow the instructions in the warning.
3. test SSH from local machine
   • if you don't have it set up already, set up passwordless ssh key-based authentication to the remote machine with the docker container
   • make SSH command that uses your remote machine as a jump server to the container: ssh -J <remote_machine> <container_ip>, as described in https://wiki.gentoo.org/wiki/SSH_jump_host; if successful you should drop into the container just as you did from the remote machine
   • save this setup in your ~/.ssh/config; follow the ProxyJump Example from https://wiki.gentoo.org/wiki/SSH_jump_host
   • test config with ssh <container_host_name_defined_in_ssh_config>; should also drop you into interactive container
4. configure pycharm (or vscode or any IDE that accepts remote SSH interpreter)
   • Preferences -> Project -> Python Interpreter -> Add -> SSH Interpreter -> New server configuration
   • host: <container_host_name_defined_in_ssh_config>
   • port: 22
   • username: <username_on_remote_server>
   • select interpreter, can navigate using the folder icon, which will walk you through paths within the docker, or you can enter the result of which python from the container
   • follow pycharm prompts