Dynamically rethrowing self-defined C++ exceptions as Python exceptions using SWIG

Question

Situation

I want to create a Python language binding for a C++ API using SWIG. Some of the API functions may throw exceptions. The C++ application has a hierarchy of self-defined exceptions, like this example:

std::exception
  -> API::Exception
    -> API::NetworkException
      -> API::TimeoutException
      -> API::UnreachableException
    -> API::InvalidAddressException

The desired behavior is as follows:

1. All exception types should have a matching Python class as wrapper. These wrapper classes should be valid Python exceptions.

2. When an API call throws a C++ exception, it should be caught. The corresponding Python exception (i.e. the wrapper class of the caught C++ exception) should be thrown.

3. This should be a dynamic process: the Python exception type is decided at runtime, only based on the runtime type of the caught C++ exception. This way, there is no need to describe the complete exception hierarchy in the SWIG interface file.

Problems and questions

1. Wrapper classes are no Python exceptions.

   While SWIG creates wrapper classes for all self-defined exceptions (like for any other class), these classes are not Python exceptions. The wrapper of the base exception (API::Exception in the example) extends Object instead of BaseException, the Python class of which all exceptions in Python should be derived.

   Furthermore, it doesn't seem possible to let SWIG add a parent class manually. Note this is possible when using SWIG with Java through the usage of %typemap(javabase) (see SWIG documentation for details).

2. How can the Python C API throw a user-defined exception?

   The most common way to throw a Python exception from the Python C API is by calling PyErr_SetString [reference]. This is also shown in the demo application below.

   But this is only trivial with the standard (built-in) exceptions of Python, because references to them are stored in global variables [reference] in the Python C API.

   I know there is a method PyErr_NewException [reference] to get references to self-defined exceptions, but I didn't got this working.

3. How can the Python C API evaluate the C++ type at runtime and then find the corresponding Python wrapper class by name?

   I assume a Python class can be searched by name at runtime, through the reflection part of the Python C API. Is this the way to go? And how is it done in practice?

Demo application

To experiment with this problem, I created a tiny C++ API with a single function that calculates the factorial of a number. It has a minimal self-defined exception hierarchy, consisting of only one class TooBigException.

Note this exception acts as the base exception in the general problem and the application should work with any subclass of it. This means the solution may only use the dynamic (i.e. runtime) type of the caught exception to rethrow it in Python (see below).

The full source code of the demo application is as follows:

// File: numbers.h
namespace numbers {
int fact(int n);
}

// File: numbers.cpp
#include "TooBigException.h"
namespace numbers {
int fact(int n) {
    if (n > 10) throw TooBigException("Value too big", n);
    else if (n <= 1) return 1;
    else return n*fact(n-1);
}
}

// File: TooBigException.h
namespace numbers {
class TooBigException: public std::exception {
public:
    explicit TooBigException(const std::string & inMessage,
                             const int inValue);
    virtual ~TooBigException() throw() {}
    virtual const char* what() const throw();
    const std::string & message() const;
    const int value() const;
private:
    std::string mMessage;
    int mValue;
};
}

// File: TooBigException.cpp
#include "TooBigException.h"
namespace numbers {
TooBigException::TooBigException(const std::string & inMessage, const int inValue):
    std::exception(),
    mMessage(inMessage),
    mValue(inValue)
{
}
const char* TooBigException::what() const throw(){
    return mMessage.c_str();
}
const std::string & TooBigException::message() const {
    return mMessage;
}
const int TooBigException::value() const {
    return mValue;
}
}

To get the Python binding I use the following SWIG interface file:

// File: numbers.i
%module numbers
%include "stl.i"
%include "exception.i"

%{
#define SWIG_FILE_WITH_INIT
#include "TooBigException.h"
#include "numbers.h"
%}

%exception {
    try {
        $action
    }
    catch (const numbers::TooBigException & e) {
        // This catches any self-defined exception in the exception hierarchy,
        // because they all derive from this base class. 
        <TODO>
    }
    catch (const std::exception & e)
    {
        SWIG_exception(SWIG_RuntimeError, (std::string("C++ std::exception: ") + e.what()).c_str());
    }
    catch (...)
    {
        SWIG_exception(SWIG_UnknownError, "C++ anonymous exception");
    }
}

%include "TooBigException.h"
%include "numbers.h"

So every call to the API is wrapped by a try-catch block. First exceptions of our base type are caught and handled. Then all other exceptions are caught and rethrown using the SWIG exception library.

Note any subclass of numbers::TooBigException is caught and the wrappers of their dynamic (i.e. runtime) type should be thrown, not the wrapper of their static (i.e. compile time) type, which is always TooBigException!

The project can be built easily by executing the following commands on a Linux machine:

$ swig -c++ -python numbers.i
$ g++ -fPIC -shared TooBigException.cpp numbers.cpp numbers_wrap.cxx \
    -I/usr/include/python2.7 -o _numbers.so

Current implementation

My current implementation still (successfully) throws a fixed standard Python exception. The code <TODO> above is then replaced by:

PyErr_SetString(PyExc_Exception, (std::string("C++ self-defined exception ") + e.what()).c_str());
return NULL;

Which gives the following (expected) behavior in Python:

>>> import numbers
>>> fact(11)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception: C++ self-defined exception Value too big

Answer 1

It looks like someone has answered your basic question over on the swig-user list...

%exception {
  try {
    $action
  } catch (MyException &_e) {
    SWIG_Python_Raise(SWIG_NewPointerObj(
            (new MyException(static_cast<const MyException& >(_e))),  
            SWIGTYPE_p_MyException,SWIG_POINTER_OWN),
        "MyException", SWIGTYPE_p_MyException); 
    SWIG_fail;
  } 
}

This does assume you have generated wrappers for your exception classes, I believe.

Answer 2

Example for your hierarchy

std::exception
  -> API::Exception
    -> API::NetworkException
      -> API::TimeoutException
      -> API::UnreachableException
    -> API::InvalidAddressException

example.i:

%module example
%include "stl.i"
%include "exception.i"

%{
#define SWIG_FILE_WITH_INIT
#include "example.cpp"
%}

%{

#define CATCH_PE(Namespace,Exception) \
    catch(const Namespace::Exception &e) \
    { \
       SWIG_Python_Raise(SWIG_NewPointerObj(new Namespace::Exception(e), \
            SWIGTYPE_p_##Namespace##__##Exception,SWIG_POINTER_OWN), \
            #Exception, SWIGTYPE_p_##Namespace##__##Exception); \
       SWIG_fail; \
    } \
/**/

// should be in "derived first" order
#define FOR_EACH_EXCEPTION(ACTION) \
   ACTION(API,UnreachableException) \
   ACTION(API,TimeoutException) \
   ACTION(API,InvalidAddressException) \
   ACTION(API,NetworkException) \
   ACTION(API,Exception) \
/**/
// In order to remove macros, need traits:
// http://swig.10945.n7.nabble.com/traits-based-access-to-swig-type-info-td12315.html
%}

%exception {
    try {
        $action
    }
    FOR_EACH_EXCEPTION(CATCH_PE)
    catch (const std::exception & e)
    {
        SWIG_exception(SWIG_RuntimeError, (std::string("C++ std::exception: ") + e.what()).c_str());
    }
    catch (...)
    {
        SWIG_exception(SWIG_UnknownError, "C++ anonymous exception");
    }
}

%include "example.cpp"

example.cpp:

#include <exception>
#include <stdexcept>

namespace API
{
    struct Exception: std::exception
    {
        virtual const char* what() const throw()
        {
            return "It is API::Exception";
        }
    };
    struct NetworkException: Exception
    {
        virtual const char* what() const throw()
        {
            return "It is API::NetworkException";
        }
    };
    struct TimeoutException: NetworkException
    {
        virtual const char* what() const throw()
        {
            return "It is API::TimeoutException";
        }
    };
    struct UnreachableException: NetworkException
    {
        virtual const char* what() const throw()
        {
            return "It is API::UnreachableException";
        }
    };
    struct InvalidAddressException: Exception
    {
        virtual const char* what() const throw()
        {
            return "It is API::InvalidAddressException";
        }
    };

    inline void select(int i)
    {
        switch(i)
        {
            case 0: throw Exception();
            case 1: throw NetworkException();
            case 2: throw TimeoutException();
            case 3: throw UnreachableException();
            case 4: throw InvalidAddressException();
            default: throw std::runtime_error("It is std::runtime_error");
        }
    }
}

Build:

swig -c++ -python example.i &&
g++ -fPIC -shared -lpython2.7 example.cpp example_wrap.cxx -I/usr/include/python2.7 -o _example.so

test.py:

#!/usr/bin/env python2.7

from exceptions import BaseException
from example import *

def catch(i):
    try:
        select(i)
    except UnreachableException as e:
        print "Caught UnreachableException"
        print e.what()
        print e
    except TimeoutException as e:
        print "Caught TimeoutException"
        print e.what()
        print e
    except InvalidAddressException as e:
        print "Caught InvalidAddressException"
        print e.what()
        print e
    except NetworkException as e:
        print "Caught NetworkException"
        print e.what()
        print e
    except Exception as e:
        print "Caught Exception"
        print e.what()
        print e
    except BaseException as e:
        print "Caught BaseException"
        print str(e)
    print "_"*16

for i in xrange(6):
    catch(i)

Output is:

Caught Exception
It is API::Exception
<example.Exception; proxy of <Swig Object of type 'API::Exception *' at 0x7f9f54a02120> >
________________
Caught NetworkException
It is API::NetworkException
<example.NetworkException; proxy of <Swig Object of type 'API::NetworkException *' at 0x7f9f54a02120> >
________________
Caught TimeoutException
It is API::TimeoutException
<example.TimeoutException; proxy of <Swig Object of type 'API::TimeoutException *' at 0x7f9f54a02120> >
________________
Caught UnreachableException
It is API::UnreachableException
<example.UnreachableException; proxy of <Swig Object of type 'API::UnreachableException *' at 0x7f9f54a02120> >
________________
Caught InvalidAddressException
It is API::InvalidAddressException
<example.InvalidAddressException; proxy of <Swig Object of type 'API::InvalidAddressException *' at 0x7f9f54a02120> >
________________
Caught BaseException
C++ std::exception: It is std::runtime_error
________________

Based on answer in maillist.