Adding custom dir to PYTHONPATH

Question

I've been trying to add a custom directory to PYTHONPATH following the advice on this post Permanently add a directory to PYTHONPATH. I'm using bash on a Mac, if that's relevant. This is what I did:

1. open ~/.bash_profile
2. export PYTHONPATH="${PYTHONPATH}:/Users/Zhengnan/Library/Python/2.7/lib/python/site-packages" and save
3. source ~/.bash_profile

There were two problems:

1. When I ran sys.path inside a Python IDE, the intended dir still didn't show up.
2. When I fired up Python in Terminal and ran sys.path there, the dir did show up, but all the other directories didn't match what I got from the previous step.

Specifically, this is what I got from running sys.path inside the IDE. The intended dir couldn't be found.

sys.path ['', '/Applications/Spyder-Py2.app/Contents/Resources', '/Applications/Spyder-Py2.app/Contents/Resources/lib/python27.zip', '/Applications/Spyder-Py2.app/Contents/Resources/lib/python2.7', '/Applications/Spyder-Py2.app/Contents/Resources/lib/python2.7/plat-darwin', '/Applications/Spyder-Py2.app/Contents/Resources/lib/python2.7/plat-mac', '/Applications/Spyder-Py2.app/Contents/Resources/lib/python2.7/plat-mac/lib-scriptpackages', '/Applications/Spyder-Py2.app/Contents/Resources/lib/python2.7/lib-tk', '/Applications/Spyder-Py2.app/Contents/Resources/lib/python2.7/lib-old', '/Applications/Spyder-Py2.app/Contents/Resources/lib/python2.7/lib-dynload', '/Applications/Spyder-Py2.app/Contents/Resources/lib/python2.7/site-packages.zip', '/Applications/Spyder-Py2.app/Contents/Resources/lib/python2.7/site-packages', '/Applications/Spyder-Py2.app/Contents/Resources/lib/python2.7/IPython/extensions', '/Users/Zhengnan/.ipython']

And this is what I got from running sys.path from Terminal. The intended dir is the third element in the list.

sys.path ['', '/Users/Zhengnan', '/Users/Zhengnan/Library/Python/2.7/lib/python/site-packages', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC']

I should mention that the reason I want to add this custom dir to PYTHONPATH is that every time I pip install a package, it gets installed in /Users/Zhengnan/Library/Python/2.7/lib/python/site-packages and I don't want to sys.path.append every time I run a script. Please advise. Thanks.

Answer 1

There's a lot going on here.

Fundamentally, the Python you are using in your IDE is not the Python you are using in the terminal. This is why pip install doesn't put things in the right place.

The easiest solution is just to modify your $PATH environment variable so that when you type python in the terminal, you get the same version that your IDE is using. My guess is that your IDE's python is something like /Applications/Spyder-Py2.app/Contents/Resources/bin/python, in which case you would get rid of your PYTHONPATH setting your .bash_profile and add:

export PATH="/Applications/Spyder-Py2.app/Contents/Resources/bin:$PATH"

Assuming that pip is available in the same place, you should now be able to pip install things without needing to muck about with PYTHONPATH.

The other problem here is that environment variables set in the shell, e.g., in your .bash_profile, have no impact on the environment visible to applications. It is possible to set environment variables that will be visible to OS X applications (e.g., see this question), but it's tricky and I wouldn't advise it.

An alternate solution, if it's available, is just to tell your IDE which python it should be using, and point it at whichever one is visible from the terminal.