Connecting Spyder to Remote Jupyter Notebook in a Docker Container

Question

I have been trying to connect Spyder to a docker container running on a remote server and failing time and again. Here is a quick diagram of what I am trying to achieve:

Currently I am launching the docker container on the remote machine through ssh with

docker run --runtime=nvidia -it --rm --shm-size=2g -v /home/timo/storage:/storage -v /etc/passwd:/etc/passwd -v /etc/group:/etc/group --ulimit memlock=-1 -p 8888:8888 --ipc=host ufoym/deepo:all-jupyter

so I am forwarding on port 8888. Then inside the docker container I am running

jupyter notebook --no-browser --ip=0.0.0.0 --port=8888 --allow-root --notebook-dir='/storage' 

OK, now for the Spyder part - As per the instructions here, I go to ~/.local/share/jupyter/runtime, where I find the following files:

kernel-ada17ae4-e8c3-4e17-9f8f-1c029c56b4f0.json  nbserver-11-open.html  nbserver-21-open.html  notebook_cookie_secret
kernel-e81bc397-05b5-4710-89b6-2aa2adab5f9c.json  nbserver-11.json       nbserver-21.json

Not knowing which one to take, I copy them all to my local machine.

I now go to Consoles->Connect to an Existing Kernel, which gives me the "Connect to an Existing Kernel" window which I fill out as so (of course using my actual remote IP address):

(here I have chosen the first of the json files for Connection info:). I hit enter and Spyder goes dark and crashes.

This happens regardless of which connection info file I choose. So, my questions are:

1: Am I doing all of this correctly? I have found lots of instructions for how to connect to remote servers, but not so far for specifically connecting to a jupyter notebook on a docker on a remote server.

2: If yes, then what else can I do to troubleshoot the issues I am encountering?

I should also note that I have no problems connecting to the Jupyter Notebook through the browser on my local machine. It's just that I would prefer to be working with Spyder as my IDE.

Many thanks in advance!

Answer 1

This isn't a solution so much as a work around, but sshfs might be of help

Use sshfs to mount the remote machine's home directory on a local directory, then your local copy of Spyder can edit the file as if it were a local file.

sshfs remotehost.com:/home/user/ ./remote-host/

It typically takes about half a second to upload the changes to an AWS host when you I hit save in Spyder, which is an acceptable delay for me. When it's time to run the code, ssh into the remote machine, and run the code from an IPython shell. It's not elegant, but it does work.

I'm not expecting this to be the best answer, but maybe you can use it as a stopgap solution.