Python 2.x multiple version issues regarding PYTHONPATH

Question

There's Python 2.6 installed in the system.

Now I want to use modules introduced in Python 2.7. Because I have no root privilege, I have built & installed 2.7 under my home directory ($HOME/local/)

I added the following to my $HOME/.bashrc:

export PATH=$HOME/local/bin:$PATH
export PYTHONPATH=$HOME/local/lib/python2.7:$PYTHONPATH

Now I encountered the two problems I want ask for workarounds.

1. Invoking Python 2.7

Newly installed Python 2.7 doesn't find 2.6 modules in system's library path (/usr/lib/python2.6/site-packages/).

Should I add it to PYTHONPATH manually? Is there any nicer solution?

2. Invoking Python 2.6

Python 2.6 complains at startup:

'import site' failed; use -v for traceback

I guess it's trying to load 2.7 modules (in $HOME/local/lib/python2.7). Is it possible to load only 2.6 modules when Python 2.6 is invoked?

Thanks.

Answer 1

1) Invoking python 2.7

In short: don't do this. There are reasons why the path is called '/usr/lib/python*2.6*/site-packages/'.

One reason is, that in this directory typically the 'compiled' python files (.pyc) are stored. python 2.6 and python 2.7 .pyc files are not compatible:

$ python2.7 /usr/lib/python2.6/sitecustomize.pyc
RuntimeError: Bad magic number in .pyc file

python will skip pyc files which it cannot understood, but you loose at least the benefits of precompiled files.

Another reason is, that things might get mixed up:

$ strace -f python2.7 /usr/lib/python2.6/sitecustomize.py
...
stat("/etc/python2.6", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat("/etc/python2.6", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat("/etc/python2.6/apport_python_hook", 0x7fffa15601f0) = -1 ENOENT (No such file or directory)
open("/etc/python2.6/apport_python_hook.so", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/etc/python2.6/apport_python_hookmodule.so", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/etc/python2.6/apport_python_hook.py", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/etc/python2.6/apport_python_hook.pyc", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/usr/lib/python2.7/apport_python_hook", 0x7fffa15601f0) = -1 ENOENT (No such file or directory)
open("/usr/lib/python2.7/apport_python_hook.so", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib/python2.7/apport_python_hookmodule.so", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib/python2.7/apport_python_hook.py", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib/python2.7/apport_python_hook.pyc", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/usr/lib/python2.7/plat-linux2/apport_python_hook", 0x7fffa15601f0) = -1 ENOENT (No such file or directory)
...

I would in your case install the modules needed also for python 2.7 in the python2.7 directory.

2) Invoking python 2.6

You might want to have a look at the part of the man page where PYTHONHOME is described:

PYTHONHOME: Change the location of the standard Python libraries. By default, the libraries are searched in ${prefix}/lib/python[version] and ${exec_prefix}/lib/python[version], where ${prefix} and ${exec_prefix} are installation-dependent directories, both defaulting to /usr/local

You can store the python 2.7 specific files / modules in the appropriate directory in your local installation. Those files / modules will only be picked up when you run the specific version of python. In this case you must not set the PYTHONPATH (or PYTHONHOME).

Note: this is exactly the way Debian (and maybe other distributions) manage different simultaneously installed versions of python.

[Edit: Added section 1 after receiving a comment from niboshi.]

Answer 2

On startup Python takes PYTHONPATH environment variable and puts it into sys.path variable. When you try to import a module it looks to the paths in sys.path

Because of:

export PYTHONPATH=$HOME/local/lib/python2.7:$PYTHONPATH

your Python 2.7 paths are in the beginning of sys.path, before the paths of Python 2.6 (You can print sys.path to check). That means that modules from $HOME/local/lib/python2.7 will have a priority.

To customize the paths for some of your scripts, either set PYTHONPATH per script, or modify sys.path (sys.path.insert(0, '/home/user/local/lib/python2.7') right in your script before any import is done.

Or copy a specific module to your project under a different name. For example i copied collections module from Python 2.7 to my project with as collections27.py, and in places where i need OrderedDict i do from collection27 import OrderedDict

Is it possible to load only 2.6 modules when Python 2.6 is invoked?

Yes, i guess. Just assure that only Python 2.6 modules are in the path - don't use:

export PYTHONPATH=$HOME/local/lib/python2.7:$PYTHONPATH