How do you assign bz2 to an altinstall of python?

Question

I'm using redhat 5.8, which comes with python 2.4 installed automatically, but I'm using a python package that requires python 2.6 or higher. SO, I installed python 2.7 alongside 2.4, so as to not step on the system version.

Now, I'm trying to install a package through pip, and get the following error:

CompressionError: bz2 module is not available

I do, however, have the module on my machine, as evidenced when I do this the server version gives:

[~]$ python -c "import bz2; print bz2.__doc__"
The python bz2 module provides a comprehensive interface for
the bz2 compression library. It implements a complete file
interface, one shot (de)compression functions, and types for
sequential (de)compression.

and the 2.7 install errors in this way:

[~]$ python2.7 -c "import bz2; print bz2.__doc__"
 Traceback (most recent call last):
  File "<string>", line 1, in <module>
 ImportError: No module named bz2

So, I have read these questions: Already installed and this very good one, but neither of these seem quite on the mark. In the first case, the advice is to install the missing piece, and the second question is to remove (or stop referencing) the extra python install.

What I want to do is put in a symlink or some such so that the python 2.7 install knows where bz2 is so that I can use pip to install a python package.

Thanks, B

---

EDIT: more information

So, after much research, it appears that the way the path variables are established changed dramatically in python 2.5 (probably why red hat hasn't updated).

So, in python 2.7 you can add to the PYTHONPATH variable by adding a file with the .pth extension in this folder:

/usr/local/lib/python2.7/site-packages/

I have tried 2 ways to get this to work properly.

First, I simply fed a number of the python 2.4 path files to 2.7. This caused an error of a different type:

[~]$ python2.7 -c "import bz2; print bz2.__doc__"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: /usr/lib64/python2.4/lib-dynload/bz2.so: undefined symbol: Py_InitModule4

So, that's something.

I also tried pointing the path to the libbz2.so file in /usr/lib/ which resulted in the familiar error:

[~]$ python2.7 -c "import bz2; print bz2.__doc__"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named bz2

I'm still stumped, but I feel like I'm closing in.

I could really use some input from someone with more experience setting up programming environments. I'm much more comfortable just writing code :)

Answer 1

Just did this yesterday on the same platform and got it working. Here's what I did:

Set CFLAGS="-I/path/to/bz2/include" and LDFLAGS="-L/path/to/bz2/lib". Make sure the bz2 library path is set in your LD_LIBRARY_PATH. You'll likely need to do a make distclean and configure && make && make install, though.

If that fails, directly edit the setup.py and make a replacement similar to the following:

# Gustavo Niemeyer's bz2 module.
if (self.compiler.find_library_file(['/home/someuser/packages/libbz2/lib'], 'bz2')): #lib_dirs, 'bz2')):

Note the commented out portion at the end of the second line is the original rest of the setup.py line.

Also, I found just downloading and building the latest version of bz2 and pointing all of the above to that easier than trying to get the system installed version to work.

Regardless, this definitely works. I did it yesterday :)

Answer 2

Any time you're working with a python other than the system-installed one, I strongly suggest using http://pypi.python.org/pypi/virtualenv. It sets up a "sandbox" for you with its own copy of python and redoes all the paths, etc. to point to the new copy. You can then use pip to install whatever packages you need.

Answer 3

The other answers provided are useful and helpful, but this is how I got this to work.

The problem wasn't that Python wasn't finding the files (as I thought), it just couldn't use them properly. So, I vi'ed into the Makefile for bzip2, found the line that looked like this:

CFLAGS= -Wall -Winline -O2 -g $(BIGFILES)

and added -fPIC to the line like so:

CFLAGS=-fPIC -Wall -Winline -O2 -g $(BIGFILES)

and WHALA! It compiled just fine.