Python* to boost::python::object

Question

I am trying to build a Python module in C++ that transforms a 2D vector into a Numpy 2D array. What is incorrect here - presumably there is some transformation needed to a boost python object from PyObject*?

boost::python::object build_day(int year, int day) {

  PyObject* arr;
  const int HEIGHT = 5;
  const int WIDTH = 5;

  std::vector<std::vector<float> > array(WIDTH, std::vector<float>(HEIGHT));

  npy_intp dims[2] = {WIDTH, HEIGHT};
  arr = PyArray_SimpleNewFromData(2, dims, NPY_FLOAT, &array);

  return arr; 
}

BOOST_PYTHON_MODULE(sumpar) {
  using namespace boost::python;
  def("build_day", build_day, args("year", "day"));
}

Answer 1

The boost::python::object provides a generalized interface to Python objects. To construct one from a PyObject*, one must first construct a boost::python::handle<>, which is essentially a smart pointer designed to manage reference-counted Python objects (PyObject* or derived types). One often uses handle<> between the boundary between Boost.Python's higher-level code and the Python/C API.

namespace python = boost::python;
PyObject* py_object = get_py_object();
python::handle<> handle(py_object);
boost::python object(handle);

Note that the handle will share ownership of the PyObject*, and during destruction, it will decrease the reference count on the PyObject it is managing. Thus, during construction, it is important to specify whether or not handle<> needs to increase the reference count of PyObject*.

If PyObject has already had its reference count increased, then use:

namespace python = boost::python;
PyObject* py_object = ...;
python::handle<> handle(py_object);
python::object object(handle);

If PyObject has not had its reference count increased, and the handle must do it, then use the borrowed() function during construction:

namespace python = boost::python;
PyObject* py_object = ...;
python::handle<> handle(python::borrowed(py_object));
python::object object(handle);

---

Here is a complete example demonstrating constructing a boost::python::object from a PyObject*:

#include <vector>
#include <boost/python.hpp>

// Mocks...
enum { NPY_FLOAT };
typedef int npy_intp;
PyObject* PyArray_SimpleNewFromData(int, npy_intp*, int, void*)
{
  return PyString_FromString("hello world");
}

boost::python::object build_day(int year, int day)
{
  const int HEIGHT = 5;
  const int WIDTH = 5;

  std::vector<std::vector<float> > array(
      WIDTH, std::vector<float>(HEIGHT));

  npy_intp dims[2] = {WIDTH, HEIGHT};

  namespace python = boost::python;
  PyObject* arr = PyArray_SimpleNewFromData(2, dims, NPY_FLOAT, &array);
  python::handle<> handle(arr);
  return python::object(handle);
}

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  python::def("build_day", &build_day, python::args("year", "day"));
}

Interactive usage:

>>> import example
>>> day = example.build_day(1, 2);
>>> assert(day)

---

Note that to create a minimal complete example, the above example has a mocked PyArray_SimpleNewFromData() that simply returns Python string. It is important to consult the documentation to determine if the PyObject* is borrowed or not, and if there are any lifetime requirements between the object and its arguments. In the case of PyArray_SimpleNewFromData(), the returned PyObject*:

• already has its reference count increased
• the lifetime of the underlying memory provided to the array must be at least as long as the returned PyObject. The build_day() function in the original question fails to meet this requirement.