The reason cause different location of python packages

Question

I am new to Linux system and don't have background about operating system.

I have some issues about importing numpy, but I worked it out.

But still have some questions in my mind.

I know that I can use following code to know where Python finds packages

import sys
sys.path()

which in my case gives several options:

/home/twotwo/.local/lib/python3.6/site-packages
/usr/local/lib/python3.6/dist-packages
/usr/lib/python3/dist-packages

And I would like to know the difference between them.

I guess that the difference between first and second is caused by the way I download it as cited below, if not please do correct me.

We recommend using an user install, using the --user flag to pip (note: do not use sudo pip, which can cause problems). This installs packages for your local user, and does not write to the system directories.

And the second and third one both have/usr in the beginning.

But there's still a difference between, one has local and the other one doesn't.

To summarize my questions:

(1) The difference between:

/home/twotwo/.local/lib/python3.6/site-packages 

/usr/local/lib/python3.6/dist-packages

(2) The difference between:

/usr/local/lib/python3.6/dist-packages

/usr/lib/python3/dist-packages

(3) Actually, two of these paths have numpy but how does python choose which one to be imported?

Many Thanks to you all, and have a nice day!

Answer 1

Your question is mainly about the Linux filesystem layout. You can read a lot about that, for example on Wikipedia and more specific for the different lib locations in this askubuntu question.

I'll try to answer (1) and (2) by summarizing how the three given folders are conventionally used:

• /usr/lib/python3/dist-packages contains non-host-specific modules installed by the system with the package manager, for example on Ubuntu with sudo apt-get install python-numpy.

• /usr/local/lib/python3.6/dist-packages contains modules that you installed yourself system-wide through a package manager, for example with sudo pip install numpy. (Of cause using sudo pip can cause problems as you rightly mentioned.)

• /home/twotwo/.local/lib/python3.6/site-packages contains modules that the user twotwo has installed in his own user directory, for example by using pip in user-mode. Those modules can of cause only be imported by twotwo, because they don't appear in other user's PATH variables and might not even be readable by another user.

Also note that the naming dist-packages is a Debian (and derivates) specific convention for python packages installed using debian packages, as explained here. In a manual python installation, the respective folders would be named site-packages, as is standard for pip.

As to question (3): The details about this can be read in the Python 3 docs. Basically, after looking for the module in the folder from which your python script is run, the folders in your sys.path variable are looked up in the same order in which they are listed there. As soon as a module of matching name is found, it is imported.