cd vs !cd vs %cd in IPython

Question

I thought prepending a shell command with a ! in IPython makes the system shell actually execute the command, but it seems this isn't the case. Consider the following, where I begin in /home/Documents/Rx, start IPython, use the command cd (with no !), then exit IPython and see which directory I am in:

$ pwd
/home/Documents/Rx
$ ipython
Python 2.7.11 |Anaconda 2.5.0 (64-bit)| (default, Dec  6 2015, 18:08:32) 
Type "copyright", "credits" or "license" for more information.

IPython 4.0.3 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: cd Papers/
/home/Documents/Rx/Papers

In [2]: pwd
Out[2]: u'/home/Documents/Rx/Papers'

In [3]: exit()
$ pwd
/home/Documents/Rx

As you can see, I did not actually change directories; only in the IPython shell. I thought this was because I didn't use !cd in IPython, but that doesn't work, either:

$ pwd
/home/Documents/Rx
$ ipython
Python 2.7.11 |Anaconda 2.5.0 (64-bit)| (default, Dec  6 2015, 18:08:32) 
Type "copyright", "credits" or "license" for more information.

IPython 4.0.3 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: !cd Papers/

In [2]: pwd
Out[2]: u'/home/Documents/Rx'

In [3]: exit()
$ pwd
/home/Documents/Rx

Note that in this case, the IPython shell didn't change directories, either; but I would have expected the actual shell to do so. Finally, using the magic command %cd has the same effect as cd. So, what's the difference between the three, and how can I actually tell the system shell to execute a command in IPython?

Answer 1

!command doesn't make "the shell" run the command. It makes a shell run the command. It's not the shell you ran IPython from; you can't send commands to that shell from IPython. It's a new shell.

When you do !cd, you launch a new shell, which changes its own current working directory and promptly shuts down. This has no effect on IPython or the shell you launched IPython from.

When you do cd or %cd, you tell IPython to change its own working directory. This will persist for the duration of your IPython session, but it still has no effect on the shell you launched IPython from. When you stop IPython, that shell will still be in whatever directory it was when you started IPython.

You can't change the parent shell's working directory from IPython.