Correctly building local python3, with bz2 support

Question

I am trying to build a local version of python3 (specifically python3.7, but same issue with 3.6.6), but am running into problems with linking to some C libraries and/or headers (at least that is what I think the problem is).

I am able to build python3.7, but run into problems with bzip2 (but I suspect there will be other system libraries that I am missing an not aware of). This manifests itself when trying to import various packages/modules that require it as an import error:

ModuleNotFoundError: No module named '_bz2'

From this stackoverflow question - Ask Question - it appears that I have built python without the right bz2 headers. The python build itself does not fail.

The specific question is how to build a local version of python with bz2? More generally, the question is how to properly link to system libraries? (since there may be others I have not encountered errors with)

In terms of what I have tried - I found this (oldish) question - How do you assign bz2 to an altinstall of python? and this more recent question Compiling bz2 support in python2 and tried a few things based off these.

First problem I ran into is actually finding the right libraries etc to link (the solution in the above question suggests to set CFLAGS="-I/path/to/bz2/include" and LDFLAGS="-L/path/to/bz2/lib"). Presumably these are on my system somewhere (ubuntu 16.04), since the system installs of python2.7 and python3.5 have no issues with bzip2.

I found libbz2.so.1 (and some others in) /lib/x86_64-linux-gnu/ - but the folders didn't seem to follow the pattern of the example/answer above. As such, I built a new local version of bz2 (in .bzp in my local folder) and then tried linking to that when I tried to build python3.7, using the following:

./configure --prefix=/home/user/.localpython CFLAGS="-I/home/user/.bz2/include" LDFLAGS="-L/home/user/.bz2/lib"

I also tried a more generic configuration (which was loosely based off Compile Python with bz2 support, linked to in the second article above.)

./configure --prefix=/home/user/.python CFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/lib"

(both of which were unsuccessful)

Any pointers, tips etc greatly appreciated - thanks in advance!

Answer 1

Ubuntu does not, by default, install the files needed to include bz2 support (or any other such library support) when compiling Python. You need to add the development headers package for the library.

For bz2 support on Ubuntu, that's the libbz2-dev package. You'll need to install more such packages for other features.

The package puts the headers in a standard location, one the Python configure script knows to find. You do not need additional flags.

Extrapolating from the Python 3.6 build dependencies on Ubuntu 17.10, you'd also want:

libbluetooth-dev
libbz2-dev
libdb-dev
libexpat1-dev
libffi-dev
libgdbm-dev
liblzma-dev
libmpdec-dev
libncursesw5-dev
libreadline-dev
libsqlite3-dev
libssl-dev
zlib1g-dev

and if you want to support the Tk GUI library, tk-dev and blt-dev.