Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Troubleshooting python sys.path

Tags:

python

path

The python docs at http://docs.python.org/library/sys.html say that sys.path is...

Initialized from the environment variable PYTHONPATH, plus an installation-dependent default.

I found a path item in my sys.path that was causing problems, and had a lot of trouble tracking it down. All I could turn up on Google was people explaining how to add items to the PYTHONPATH variable.

My question is: are there any tools that can help track down why a particular item is on your sys.path? How can I find out more about the "installation-dependent default"?

So far, I've found a partial answer is to use strace on python itself and look for .pth files. I also found a sys.path_importer_cache, which may or may not be applicable.

like image 367
Mu Mind Avatar asked Mar 31 '11 13:03

Mu Mind


People also ask

What does Sys path do in Python?

sys. path is a built-in variable within the sys module. It contains a list of directories that the interpreter will search in for the required module. When a module(a module is a python file) is imported within a Python file, the interpreter first searches for the specified module among its built-in modules.

How do I check my Python path?

Right-click on the Python App, and then select “Open file location“ Right-click on the Python shortcut, and then select Properties. Click on “Open File Location“

Why can't Python find my modules?

This is caused by the fact that the version of Python you're running your script with is not configured to search for modules where you've installed them. This happens when you use the wrong installation of pip to install packages.

How is Python SYS path populated?

How sys. path gets populated. As the docs explain, sys. path is populated using the current working directory, followed by directories listed in your PYTHONPATH environment variable, followed by installation-dependent default paths, which are controlled by the site module.


1 Answers

I had some issues recently with sys.path and here is how I went about trying to figure out where the entries are coming from. I was able to track all the entries and where they were coming from. Hopefully this will help you too.

  • The first that is added C:\WINNT\system32\python27.zip (more details in PEP273).

  • Next ones that are added are from entries in windows registry. The entries C:\Python27\DLLs;C:\Python27\lib; C:\Python27\lib\plat-win; C:\Python27\lib\lib-tk come from HOT_KEY_LOCAL_USER/Python/PythonCore/2.7/PythonPath in the registry. More details in Python source code comments here http://svn.python.org/projects/python/trunk/PC/getpathp.c (These entries were the trickiest for me to understand until I found the link above).

  • Next, as explained in the site package documentation (link), sys.path is built from sys.prefix and sys.exec_prefix. On my computer both of them point to C:\Python27. And by default it searches the lib/site-packages anywways. So now the entries C:\Python27; C:\Python27\lib\site-packages are appended to the list above.

  • Next it searches each of the .pth files in alphabetical order. I have easy_install.pth, pywin32.pth and setuptools.pth in my site-packages. This is where things start getting weird. It would be straightforward if the entries in the .pth files were just directory locations. They would just get appended to the sys.path line by line. However easy_install.pth has some python code that causes the entries listed in easy_install.pth to add the packages list at the beginning of the sys.path list.

  • After this the directory entries in pywin32.pth, setuptools.pth are added at the end of the sys.path list as expected.

Note: While the above discussion pertains to Windows, it is similar even on Mac etc. On Mac it just adds different OS defaults like darwin etc. before it starts looking at site-packages directory for .pth files.

In your case, you can start by starting a python shell and checking where sys.prefix and sys.exec_prefix point to and then drilling down from there.

Note 2: If you are using an IDE such as Aptana/PyDev it will add more configurations of its own. So you need to be wary of that.

like image 58
Praveen Gollakota Avatar answered Oct 18 '22 23:10

Praveen Gollakota