Reconnecting remote Jupyter Notebook and get current cell output

Question

I'm currently training a neural network on a remote server, using jupyter notebook. I set it up with the following:

• tmux
• jupyter-notebook --no-browser --port=5000
• connecting to jupyter notebook with a browser and executing the cell for the training (output was fine when I watched for the first 10 minutes)
• detach tmux (ctrl-b, d) and closing the browser tab

Now, when I reconnect to the jupyter notebook in the browser, I don't see the current output of the training cell, only the output that I saw when I was watching the first 10 minutes of training.

I tried to find a solution for this and, I think, there are some git issues for this certain problem but they are old and I couldn't figure out if this issue was solved or not.

edit// to make my intentions more clear, since I found some threads on StackOverflow that are addressing this problem: I don't want to wait for the training to complete, as I might want to kill the training before it finishes, when it absolutely doesn't go they way I would expect it to go. So some sort of 'live' output or at least regular output would be nice.

Answer 1

And if you use a .py file instead of a .ipynb file (jupyter notebook), and inside this .py file you print the results to test the operation of your code.

To convert from .ipynb to .py file you can use this command:

'jupyter nbconvert --to script example.ipynb' 

Now, you can work with a python script instead a jupyter notebook file, this will make things easier.

In your script write prints() in the stages you think necessary in order that you can see it in Tmux terminal. So you can kill your training whenever you want (ctr+c) or not, Tmux can save the session if you want, just tape 'ctr-b + d' to detach from de session