Fail to change directory (shell command) in jupyternotebook (or jupyterlab session)

Question

I can run shell commands from an open jupyterlab (or jupyternotebook) session with an exclamation mark preprended to the shell command, as follows:

!mkdir /new_folder

This, as well as other commands such as ls and pwd work, though if I try to change directory with cd, as shown below

!cd /path/to/mydir

this does not work and I have noticed that the current working directory will always be the one where my jupyter notebook (.ipynb) is saved.

It is also odd that if I do:

!cd /path/to/mydir && pwd

I will get /path/to/mydir printed out, though if, on the cell below I do

!pwd

I will get the current directory where my jupyternotebook is saved i.e. apparently I eventually cannot change the working directory with !cd in a jupyternotebook.

Does anyone know what the problem could be?

Answer 1

You cannot use !cd to navigate the filesystem from Jupyter notebook. The reason is that shell commands (preceding with ! symbol) in Jupyter notebook's code cells are executed in a temporary subshell. If you'd like to change the working directory, you can use the %cd magic command:

!pwd
/d/swatchai_works/tutorial/jupyter

%cd ..
/d/swatchai_works/tutorial

!pwd
/d/swatchai_works/tutorial

%cd jupyter
/d/swatchai_works/tutorial/jupyter

Edit Using magic command %cd to change working directory can cause problems to other code running on the jupyter notebook that rely on the directory at the startup of the notebook.

Using chaining bash commands with "&&" operator(s) may be better choice if you dont want to change the current working directory at the end of the process.