Getting Pypy to recognize third party modules

Question

Just a quick question, how do I get pypy to recognize third pary modules that I have in Python? For instance, I get the following error.

from tables import *
ImportError: No Module named tables

Which is basically saying that it cannot find my pytables library that I use to interact with in the script I am trying to run.

Answer 1

For pure python modules, just add the directory containing the modules to your sys.path, using something like:

sys.path.insert(0, '/usr/local/lib')
sys.path.insert(0, os.path.expanduser('~/lib'))

This works for CPython, Pypy and Jython.

For C extension modules, you can try Pypy's cpyext, but it won't run everything you might hope for, because some CPython C extension modules wander into dark corners of CPython's C-based runtime: http://morepypy.blogspot.com/2010/04/using-cpython-extension-modules-with.html

I sometimes write code that uses ctypes to interface with a C .so, and then use that on both CPython and Pypy, because they both do pretty well with ctypes - but ctypes can be kinda slow on CPython: http://docs.python.org/library/ctypes.html Last I checked, Jython had the beginnings of ctypes, but it wasn't far enough along to use, at least not for my purposes.

There's also a new interface that requires a C compiler at runtime. It'll likely be less brittle (read: prone to segfaults) than ctypes. It's described here: http://morepypy.blogspot.com/2012/06/release-01-of-cffi.html It comes from the Pypy project I believe, but it was made to work first on CPython. AFAIK, it doesn't yet run on Pypy.

Answer 2

Pypy has a separate install space. Therefore, any modules you want to install from pypi should be installed into its space. So, for instance, I have pypy installed in /usr/local/pypy-1.9-32bit

I recommend using pip or easy_install. Here's how to install pip:

curl curl https://bootstrap.pypa.io/get-pip.py | /usr/local/pypy-1.9-32bit/bin/pypy

or

curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | /usr/local/pypy-1.9-32bit/bin/pypy

Then, just use the newly installed pip to get the module:

sudo /usr/local/pypy-1.9-32bit/bin/pip install tables

In this case, it failed, with the following error:

bminton@bminton:/tmp$ sudo /usr/local/pypy-1.9-32bit/bin/pip install tables
Downloading/unpacking tables
  Downloading tables-2.4.0.tar.gz (8.9Mb): 8.9Mb downloaded
  Running setup.py egg_info for package tables
    .. ERROR:: You need numpy 1.4.1 or greater to run PyTables!
    Complete output from command python setup.py egg_info:
    .. ERROR:: You need numpy 1.4.1 or greater to run PyTables!

Installation failed in this case, because Tables depends on Numpy, which is not yet supported by PyPy (although they are working on it, see http://pypy.org/numpydonate.html). However, for many python modules, this method works great. For instance, I successfully installed the logilab constraint package this way.