Python + setuptools: distributing a pre-compiled shared library with boost.python bindings

Question

I have a C++ library (we'll call it Example in the following) for which I wrote Python bindings using the boost.python library. This Python-wrapped library will be called pyExample. The entire project is built using CMake and the resulting Python-wrapped library is a file named libpyExample.so.

When I use the Python bindings from a Python script located in the same directory as libpyExample.so, I simply have to write:

import libpyExample
libpyExample.hello_world()

and this executes a hello_world() function exposed by the wrapping process.

What I want to do

For convenience, I would like my pyExample library to be available from anywhere simply using the command

import pyExample

I also want pyExample to be easily installable in any virtualenv in just one command. So I thought a convenient process would be to use setuptools to make that happen. That would therefore imply:

• Making libpyExample.so visible for any Python script
• Changing the name under which the module is accessed

I did find many things about compiling C++ extensions with setuptools, but nothing about packaging a pre-compiled C++ extension. Is what I want to do even possible?

What I do not want to do

I don't want to build the pyExample library with setuptools, I would like to avoid modifying the existing project too much. The CMake build is just fine, I can retrieve the libpyExample.so file very easily.

Answer 1

If I understand your question correctly, you have the following situation:

• you have an existing CMake-based build of a C++ library with Python bindings
• you want to package this library with setuptools

The latter then allows you to call python setup.py install --user, which installs the lib in the site-packages directory and makes it available from every path in your system.

What you want is possible, if you overload the classes that setuptools uses to build extensions, so that those classes actually call your CMake build system. This is not trivial, but you can find a working example here, provided by the pybind11 project:

https://github.com/pybind/cmake_example

Have a look into setup.py, you will see how the classes build_ext and Extension are inherited from and modified to execute the CMake build.

This should work out of the box for you or with little modification - if your build requires special -D flags to be set. I hope this helps!