Exposing C++ functions, that return pointer using Boost.Python

Question

I want to expose the following C++ function to Python using Boost.Python:

int* test1() {
    return new int(42);
}

// Now exposing the function with Boost.Python

BOOST_PYTHON_MODULE(libtest1)
{
    using namespace boost::python;
    def("test1", test1);
}

When I try to compile this library the error occurs due to (it's my guess) Boost.Python don't know, how to convert int* to PyObject.

I think what needs to be done is to define conversion structure, something like this:

template<class T>
struct int_ptr_to_python
{
   static PyObject* convert(int* i_ptr)
   {
        return i_ptr;
   }
};

And pass it to the BOOST_PYTHON_MODULE declaration:

BOOST_PYTHON_MODULE(libtest1)
{
    using namespace boost::python;
    def("test1", test1);
    to_python_converter<int*, int_ptr_to_python<int*> >();
}

But it also doesn't work. And I can't find any information about how the functions, that return pointers should be handled.

Does anyone can help?

Answer 1

In short, one cannot directly expose a function returning int* with Boost.Python, as there is no meaningful corresponding type in Python given integers are immutable.

• If the goal is to return a number to Python, then return int by value. This may require using an auxiliary function to adapt legacy APIs.
• If the goal is to return a pointer to an object allocated with a new-expression, then the object must be a class or union, and the function must be exposed with specific policies to indicate ownership responsabilities.

---

Rather than present the final solution immediately, I would like to take the time to step through the compiler errors. With Boost.Python, sometimes pre-C++11 static assertions are used to provide instructions in the compiler messages. Unfortunately, it can be a bit difficult to find them amongst the heavy templates.

The following code:

#include <boost/python.hpp>

int* make_int() { return new int(42); }

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  python::def("make_int", &make_int);
}

produces the following relevant output in clang, with the important details accentuated in bold:

.../boost/python/detail/caller.hpp:102:98: error:
  no member named 'get_pytype' in 'boost::python::detail::
    specify_a_return_value_policy_to_wrap_functions_returning<int*>'
  ...create_result_converter((PyObject*)0, (ResultConverter *)0, 
                             (ResultConverter *)0).g...

Boost.Python is informing us that a boost::python::return_value_policy needs to be specified for functions returning int*. There are various models of ResultConverterGenerators. Often times the policies are used to control the ownership or lifetime semantics of the returned object. As the function in the original code is returning a new pointer directly to Python, the boost::python::manage_new_object is appropriate if the caller is expected to take responsibility for deleting the object.

Specifying a policy for object management still fails:

#include <boost/python.hpp>

int* make_int() { return new int(42); }

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  python::def("make_int", &make_int,
    python::return_value_policy<python::manage_new_object>());
}

produces the following relevant output:

.../boost/python/object/make_instance.hpp:27:9:
  error: no matching function for call to 'assertion_failed'
    BOOST_MPL_ASSERT((mpl::or_<is_class<T>, is_union<T> >));

In this case, Boost.Python is informing us that the object returned from a function with a managed_new_object ResultConverterGenerator must be either a class or union. For an int*, the most appropriate solution is to return the int by value when passing through the Boost.Python layer. However, for completeness, below demonstrates:

• Using an auxiliary function to adapt a legacy API.
• How to limit the creation of a user defined type with a factory function that returns pointers.

#include <boost/python.hpp>

/// Legacy API.
int* make_int() { return new int(42); }

/// Auxiliary function that adapts the legacy API to Python.
int py_make_int()
{
  std::auto_ptr<int> ptr(make_int());
  return *ptr;
}

/// Auxiliary class that adapts the legacy API to Python.
class holder
  : private boost::noncopyable
{
public:
  holder()
    : value_(make_int())
  {}

  int get_value() const     { return *value_; }
  void set_value(int value) { *value_ = value; }

private:
  std::auto_ptr<int> value_;
};

/// Factory function for the holder class.
holder* make_holder()
{
  return new holder();
}

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  python::def("make_int", &py_make_int);

  python::class_<holder, boost::noncopyable>("Holder", python::no_init)
    .add_property("value",
      python::make_function(&holder::get_value),
      python::make_function(&holder::set_value))
    ;

  python::def("make_holder", &make_holder,
    python::return_value_policy<python::manage_new_object>());
}

Interactive usage:

>>> import example
>>> assert(42 == example.make_int())
>>> holder = example.Holder()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: This class cannot be instantiated from Python
>>> holder = example.make_holder()
>>> assert(42 == holder.value)
>>> holder.value *= 2
>>> assert(84 == holder.value)