How to Implement Python Interfaces for C++ Libraries

Question

What is the best/standard way to create Python interfaces for C++ libraries?

I know this question has been asked on here before but that was in 2008 and things may/likely have changed since then.

I've looked around and tested a few different methods but can't decide which is best. I've tried Swig, ctypes, and cppyy so far and think that cppyy is by far the easiest/fastest to implement. I've seen recommendations for Swig but it took a very long time to get Swig working and the results were not impressive. Is there a current standard? Why do people recommend Swig so much but I hear nothing of cppyy? Thank you.

Answer 1

SWIG has been around since February 1996 and supports a range of languages, not just Python. Although what is now cppyy started in February 2003, as RootPython, it has always been embedded with ROOT (http://root.cern.ch), and was not available standalone. A full, easy, installation with wheels on PyPI for all three major platforms only exists since March of this year and on conda-forge (for Linux & Mac) only since two months now. So, even though it has a long pedigree, within the wider Python world cppyy is really quite fresh, which is why I doubt many folks have heard of it yet, whereas SWIG is the (spiritual) ancestor of them all.

The reason for putting in the effort of making cppyy available, is that it offers quite a few features that other binders do not have, and would not be easy to add: a compliant C++17 parser (b/c of Clang/LLVM); automatic template instantiations, cross-inheritance and callbacks, all at run-time (b/c of Cling); and much better performance. It also does not create C extension modules, so you only need to recompile cppyy itself for different versions of Python, but none of the bound code.

Now, to your first question of what is the best. Well, it depends on the use case. For example, if you need more bindings than just Python, SWIG is your best bet. If you have lots of templates that you can not all instantiate at build time, need the performance and scaling, or have a C++ framework with lots of interfaces, then cppyy is hard to beat. If you have modern C++ and do not want any run-time dependency on external libraries, then PyBind11 is where it's at.

These days I can't recommend ctypes. The only real benefit is that it is a builtin module for most Pythons in the wild, but with the advent of PyPI and conda that has become moot. If you want a super-light C binder (not C++, but you can wrap those functions with C helpers), then go for CFFI.

As to your question of whether there is a standard: no, there is no one binder that is best for all use cases. There are even quite a few more than the ones you have mentioned, but many of them play in the same space (eg. SWIG vs. SIP, and PyBind11 vs. boost.python) and I wouldn't recommend them over the ones you already tried. I do want to point out AutoWIG, which is a generator utilizing Clang with PyBind11 or boost.python code as output; and cython, which is a Python-like code to write C extension modules and which has some (limited) C++ support. I've always felt that cython was neither here nor there, but lots of people like it, and it's used extensively in the scientific community and in math-heavy code, so that vouches for its quality.

Now, even though there is no "standard", all binders can convert proxies to PyCapsule objects and rebind them. So although it is a bit clunky at times, you can actually mix binders within one application.

One final point: CFFI and cppyy (through CFFI's backend) have near-native performance on PyPy. Unfortunately, cppyy isn't as up-to-date on PyPy as it is on CPython (e.g. cross-inheritance is still missing), but it's getting there. The other binders work through the Python C-API, which is fully functional on PyPy, but precludes the JIT from doing its work, with reduced performance as result.

Full disclaimer: I'm the author of cppyy, and these days I only use cppyy, CFFI, and PyBind11 for my binding needs.