Problems compiling Python 3.2 and 2.7 using pythonbrew

Question

I'm trying to build multiple versions of python using Pythonbrew, but I'm getting some test failures. This is on a VM running: Ubuntu 8.04 32bit

This happens when I use the --enable-shared option

The build.log file errors are shown below:

Python 3.2

Here are the failed tests for Python 3.2:

LD_LIBRARY_PATH=/opt/standingcloud/stack/pythonbrew/build/Python-3.2: ./python -E -c 'import sys ; from sysconfig import get_platform ; print(get_platform()+"-"+sys.version[0:3])' >platform
find ./Lib -name '*.py[co]' -print | xargs rm -f
LD_LIBRARY_PATH=/opt/standingcloud/stack/pythonbrew/build/Python-3.2: ./python -Wd -E -bb  ./Lib/test/regrtest.py -l 
== CPython 3.2 (r32:88445, Jun 16 2011, 15:07:10) [GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu4)]
==   Linux-2.6.18.8-linode22-i686-with-debian-lenny-sid little-endian
==   /opt/standingcloud/stack/pythonbrew/build/Python-3.2/build/test_python_18839
Testing with flags: sys.flags(debug=0, division_warning=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=1, verbose=0, bytes_warning=2, quiet=0)

....SNIP....

[197/349] test_os
test test_os failed -- Traceback (most recent call last):
  File "/opt/standingcloud/stack/pythonbrew/build/Python-3.2/Lib/test/test_os.py", line 646, in test_exist_ok_existing_directory
    os.makedirs(path, mode=mode, exist_ok=True)
  File "/opt/standingcloud/stack/pythonbrew/build/Python-3.2/Lib/os.py", line 152, in makedirs
    mkdir(name, mode)
OSError: [Errno 17] File exists: '@test_18839_tmp/dir1'

....SNIP....

321 tests OK.
1 test failed:
    test_os
27 tests skipped:
    test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp
    test_codecmaps_kr test_codecmaps_tw test_curses test_dbm_gnu
    test_dbm_ndbm test_gdb test_ioctl test_kqueue test_ossaudiodev
    test_pep277 test_smtpnet test_socketserver test_sqlite
    test_startfile test_timeout test_tk test_ttk_guionly
    test_unicode_file test_urllib2net test_urllibnet test_winreg
    test_winsound test_xmlrpc_net test_zipfile64
5 skips unexpected on linux2:
    test_dbm_gnu test_dbm_ndbm test_ioctl test_tk test_ttk_guionly
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='a' encoding='ANSI_X3.4-1968'>
make: *** [test] Error 1

Python 2.7.1

Here are the failed tests for Python 2.7.1:

./python -Wd -3 -E -tt  ./Lib/test/regrtest.py -l 
== CPython 2.7.1 (r271:86832, Jun 16 2011, 17:59:28) [GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu4)]
==   Linux-2.6.18.8-linode22-i686-with-debian-lenny-sid little-endian
==   /opt/standingcloud/stack/pythonbrew/build/Python-2.7.1/build/test_python_19286

....SNIP....

test test_file2k failed -- Traceback (most recent call last):
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7.1/Lib/test/test_file2k.py", line 219, in testStdin
    self.assertRaises(IOError, sys.stdin.seek, -1)
AssertionError: IOError not raised

....SNIP....

343 tests OK.
2 tests failed:
    test_file test_file2k
41 tests skipped:
    test_aepack test_al test_applesingle test_bsddb test_bsddb185
    test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk
    test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
    test_dbm test_gdb test_gdbm test_gl test_imgfile test_ioctl
    test_kqueue test_linuxaudiodev test_macos test_macostools
    test_ossaudiodev test_pep277 test_scriptpackages test_smtpnet
    test_socketserver test_sqlite test_startfile test_sunaudiodev
    test_timeout test_tk test_ttk_guionly test_unicode_file
    test_urllib2net test_urllibnet test_winreg test_winsound
    test_zipfile64
8 skips unexpected on linux2:
    test_bsddb test_bsddb3 test_dbm test_gdb test_gdbm test_ioctl
    test_tk test_ttk_guionly
make: *** [test] Error 1

Python 2.7

Here are the failed tests for Python 2.7:

LD_LIBRARY_PATH=/opt/standingcloud/stack/pythonbrew/build/Python-2.7: ./python -E -c 'import sys ; from sysconfig import get_platform ; print get_platform()+"-"+sys.version[0:3]' >platform
find ./Lib -name '*.py[co]' -print | xargs rm -f
LD_LIBRARY_PATH=/opt/standingcloud/stack/pythonbrew/build/Python-2.7: ./python -Wd -3 -E -tt ./Lib/test/regrtest.py -l 
== CPython 2.7 (r27:82500, Jun 16 2011, 15:25:09) [GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu4)]
==   Linux-2.6.18.8-linode22-i686-with-debian-lenny-sid little-endian
==   /opt/standingcloud/stack/pythonbrew/build/Python-2.7/build/test_python_13541

....SNIP....

test_distutils
/usr/bin/ld: cannot find -lpython2.7
collect2: ld returned 1 exit status
test test_distutils failed -- Traceback (most recent call last):
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/distutils/tests/test_build_ext.py", line 269, in test_get_outputs
    cmd.run()
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/distutils/command/build_ext.py", line 449, in build_extensions
    self.build_extension(ext)
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/distutils/command/build_ext.py", line 531, in build_extension
    target_lang=language)
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/distutils/ccompiler.py", line 741, in link_shared_object
    extra_preargs, extra_postargs, build_temp, target_lang)
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/distutils/unixccompiler.py", line 258, in link
    raise LinkError, msg
LinkError: command 'gcc' failed with exit status 1

....SNIP....

test test_file failed -- multiple errors occurred; run in verbose mode for details
test_file2k
test test_file2k failed -- Traceback (most recent call last):
  File "/opt/standingcloud/stack/pythonbrew/build/Python-2.7/Lib/test/test_file2k.py", line 211, in testStdin
    self.assertRaises(IOError, sys.stdin.seek, -1)
AssertionError: IOError not raised

....SNIP....

339 tests OK.
3 tests failed:
    test_distutils test_file test_file2k
1 test altered the execution environment:
    test_site
41 tests skipped:
    test_aepack test_al test_applesingle test_bsddb test_bsddb185
    test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk
    test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
    test_dbm test_gdb test_gdbm test_gl test_imgfile test_ioctl
    test_kqueue test_linuxaudiodev test_macos test_macostools
    test_ossaudiodev test_pep277 test_scriptpackages test_smtpnet
    test_socketserver test_sqlite test_startfile test_sunaudiodev
    test_timeout test_tk test_ttk_guionly test_unicode_file
    test_urllib2net test_urllibnet test_winreg test_winsound
    test_zipfile64
8 skips unexpected on linux2:
    test_bsddb test_bsddb3 test_dbm test_gdb test_gdbm test_ioctl
    test_tk test_ttk_guionly
make: [test] Error 1 (ignored)

Answer 1

Currently, I manage to get pythonbrew working with 2.7.2 doing the following:

a) First, install all the dependencies needed to compile python.

'curl' # not for build, but for steps after
'build-essential',
'libbz2-dev',
'libsqlite3-dev',
'zlib1g-dev',
'libxml2-dev',
'libxslt1-dev',
'libreadline5', # lenny
'libreadline5-dev', # lenny
'libgdbm-dev',
'libgdb-dev',
'libxml2',
'libssl-dev',
'tk-dev',
'libgdbm-dev',
'libexpat1-dev',
'libncursesw5-dev'

b) Now install 2.7.2:

pythonbrew install --configure="--with-threads --enable-shared" -j2 -v 2.7.2

This installs, but is hit by the following bug:

/home/python-deploy/.pythonbrew/pythons/Python-2.7.2/bin/python: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory

*To avoid getting an error after the build when it tries to install setuptools, use the --no-setuptools option, and you might need to use --force on some platforms, as the gdb test fails due to an error in the test itself.

pythonbrew install --configure="--with-threads--enable-shared" \
                   --force \
                   --no-setuptools \
                   --jobs=2 \
                   --verbose 2.7.2 

c) The easiest workaround is to do (with the correct path to your pythonbrew install):

pythonbrew use 2.7.2
export LD_LIBRARY_PATH=$HOME/.pythonbrew/pythons/Python-2.7.2/lib

or to add it permanently, put the pythonbrew lib path in a file placed under /etc/ld.so.conf.d and the path will be added at startup, or right away running ldconfig:

sudo echo /home/user/.pythonbrew/pythons/Python-2.7.2/lib >> /etc/ld.so.conf.d/pythonbrew.conf
sudo ldconfig

d) Install setuptools or distribute manually, i.e.

curl -O http://python-distribute.org/distribute_setup.py
python distribute_setup.py && easy_install pip

And you should be ready to go. Only tested with 2.7.2 on a vagrant debian lenny VM.