Creating a secondary site-packages directory (and loading packages from .pth files therein)

Question

I would like to install some packages into a third-party site-packages directory (beyond the standard system locations). Is there any way to set this up such that .pth files therein are respected?

---

Background: I'm using OS X, virtualenv, and homebrew. There are a few packages (notably wxPython in my case) that do not install nicely through pip into virtualenv. In these cases, there are homebrew packages that work in their stead. Homebrew creates a third site-packages folder in /usr/local/lib/python2.7. I'd like to simply point to this folder and leave the maintenance of all items there under brew's control. It seems, however, that this is not possible.

• Appending to the path via $PYTHONPATH does not load .pth files. (Should Python 2.6 on OS X deal with multiple easy-install.pth files in $PYTHONPATH?)
• Python does not allow nesting or recursive use of .pth files. (Nested .pth Files or Loading Extra site-dirs from a Network)
• The wrapper for virtualenv add2virtualenv adds some extra smarts to the .pth file beyond simply having the directory listed, but I believe it simply reorders the path list.

I'm certainly not the only one interested in this issue. I'd wager a good number of the generic 'pth files not working' questions and posts online that I've stumbled across are related to this issue. Is there a good solution?

Answer 1

Take a look at the site module. It provides the function addsitedir which should do what you want.

The easiest way to use this would be to create a file named sitecustomize.py or usercustomize.py and place it in a current PYTHONPATH directory (or any directory that ends up on sys.path) with the following contents:

import site
site.addsitedir('/usr/local/lib/python2.7')

When Python is starting an attempt is made to import sitecustomize and then usercustomize, which is why this works. From the site documentation:

After these path manipulations, an attempt is made to import a module named sitecustomize, which can perform arbitrary site-specific customizations. It is typically created by a system administrator in the site-packages directory. If this import fails with an ImportError exception, it is silently ignored.

After this, an attempt is made to import a module named usercustomize, which can perform arbitrary user-specific customizations, if ENABLE_USER_SITE is true. This file is intended to be created in the user site-packages directory (see below), which is part of sys.path unless disabled by -s. An ImportError will be silently ignored.

Answer 2

There was PEP 370 specifically addressing the creation of per-user site-packages directories, to deal with the situation where the user has no admin access to the system-wide site-packages.

For example on Unix (including Mac OS), and assuming one is using Python 3.6, one can create the following directory and place .pth files inside there

~/.local/lib/python3.6/site-packages