Handling custom C++ exceptions in Cython

Question

I have some trouble handling custom C++ exceptions when calling from Cython. My situation is the following: I have a library that uses CustomLibraryException for all exceptions. What I want is basically get the error message and raise a Python error with it.

The user guide has some hints but it is a bit unspecific. The first possibility is to do:

cdef int bar() except +ValueError

This converts the CustomLibraryException to a ValueError, but loses the error message.

The other possibility is to explicitly convert the error using

cdef int raise_py_error()
cdef int something_dangerous() except +raise_py_error

I don't really understant this solution. I understood that raise_py_error has to be a C++ function that somehow handles the error. I am not sure how to handle it though. The function doesn't get an argument and is called inside the catch block in C++.

If any one has an working example of handling a C++ exception in Cython, that would be of great help.

Answer 1

Agreed the wording in the doc page leaves something to be desired. While "Cython cannot throw C++ exceptions", here is a raise_py_error that does what we want.

First, define the custom exception class in cython and make a handle to it using the "public" keyword

from cpython.ref cimport PyObject

class JMapError(RuntimeError):
  pass

cdef public PyObject* jmaperror = <PyObject*>JMapError

Then write the exception handler (the docs aren't super clear this must be written in C++ and imported):

#include "Python.h"
#include "jmap/cy_utils.H"
#include "jmap/errors.H"
#include <exception>
#include <string>

using namespace std;

extern PyObject *jmaperror;

void raise_py_error()
{
  try {
    throw;
  } catch (JMapError& e) {
    string msg = ::to_string(e.code()) +" "+ e.what();
    PyErr_SetString(jmaperror, msg.c_str());
  } catch (const std::exception& e) {
    PyErr_SetString(PyExc_RuntimeError, e.what() );
  }
}

Finally, bring the handler into cython with an extern block, and use it:

cdef extern from "jmap/cy_utils.H":
  cdef void raise_py_error()

void _connect "connect"() except +raise_py_error

Done. I now see new exception, constructed with the error code as intended:

JMapError: 520 timed connect failed: Connection refused

Answer 2

The default C++ exception handler in Cython should illustrate exactly how to accomplish what you are trying to do:

static void __Pyx_CppExn2PyErr() {
  // Catch a handful of different errors here and turn them into the
  // equivalent Python errors.
  try {
    if (PyErr_Occurred())
      ; // let the latest Python exn pass through and ignore the current one
    else
      throw;
  } catch (const std::bad_alloc& exn) {
    PyErr_SetString(PyExc_MemoryError, exn.what());
  } catch (const std::bad_cast& exn) {
    PyErr_SetString(PyExc_TypeError, exn.what());
  } catch (const std::domain_error& exn) {
    PyErr_SetString(PyExc_ValueError, exn.what());
  } catch (const std::invalid_argument& exn) {
    PyErr_SetString(PyExc_ValueError, exn.what());
  } catch (const std::ios_base::failure& exn) {
    // Unfortunately, in standard C++ we have no way of distinguishing EOF
    // from other errors here; be careful with the exception mask
    PyErr_SetString(PyExc_IOError, exn.what());
  } catch (const std::out_of_range& exn) {
    // Change out_of_range to IndexError
    PyErr_SetString(PyExc_IndexError, exn.what());
  } catch (const std::overflow_error& exn) {
    PyErr_SetString(PyExc_OverflowError, exn.what());
  } catch (const std::range_error& exn) {
    PyErr_SetString(PyExc_ArithmeticError, exn.what());
  } catch (const std::underflow_error& exn) {
    PyErr_SetString(PyExc_ArithmeticError, exn.what());
  } catch (const std::exception& exn) {
    PyErr_SetString(PyExc_RuntimeError, exn.what());
  }
  catch (...)
  {
    PyErr_SetString(PyExc_RuntimeError, "Unknown exception");
  }
}

So you can either #define __Pyx_CppExn2PyErr your_custom_exn_handler in an included .h file to override the generic behavior, or use a one-off custom handler as

cdef extern from "...":
    void your_exn_throwing_fcn() except +your_custom_exn_handler

Answer 3

If CustomLibraryException derives from std::runtime_error (as a well-behaved C++ exception should), then the behavior you're seeing is a bug in Cython.

If it doesn't, then the easiest thing to do is to wrap the C++ function you're calling in a C++ helper function that translates the exception:

double foo(char const *, Bla const &);  // this is the one we're wrapping

double foo_that_throws_runtime_error(char const *str, Bla const &blaref)
{
    try {
        return foo(str, blaref);
    } catch (CustomLibraryException const &e) {
        throw std::runtime_error(e.get_the_message());
    }
}

This will cause a RuntimeError to be raised on the Python side. Alternatively, throw an std::invalid_argument to raise a ValueError, etc. (see the table in the page you linked to).

Answer 4

In Cython's sources, https://github.com/cython/cython/blob/master/tests/run/cpp_exceptions.pyx they actually implement the raise_py_error in a .pyx file. This makes it a lot easier to share error handling between other .pyx files.

A quick solution involves simply 2 files : myerror.pyx :

class MyError(RuntimeError):
    "Base class for errors raised from my C++."
    pass
cdef int raise_my_py_error() except *:
    raise MyError("There was a problem")

and myerror.pxd :

cdef int raise_my_py_error() except *

which allows you to add an except +my_py_error in all your files.

However, this "loses" the e.what() of C++ exceptions. So a more interesting solution needs a couple more helper files :

my_error_helper.h :

extern const char* get_my_py_error_message();

my_error_helper.cxx :

#include <exception>
const char* get_my_py_error_message()
{
  try {
    throw;
  } catch (const my_custom_cxx_exception& e) {
    return e.what();
  }
}

my_error_helper.pxd :

cdef extern from "my_error_helper.h": 
    const char* get_my_py_error_message()

my_error.pxd :

cdef int raise_my_py_error() except *

my_error.pyx :

cimport my_error_helper

class MyError(RuntimeError):
    "Base class for errors raised from my C++."
    pass

cdef int raise_my_py_error() except *:
    msg = my_error_helper.get_my_py_error_message().decode('utf-8')
    raise MyError(msg)