How to pass non-const std::vector<double> by reference to Python with boost::python?

Question

I would like to be able to give an empty std::vector<double>& as argument of a Python function able to fill it. Something like that :

Python file Foo.py :

class Foo :
  def bar(self, x) :
    x.push_back(3.14)
foo = Foo()

C++ code :

Py_Initialize();
boost::python::object pyobj_main = boost::python::import("__main__");
boost::python::object glob = pyobj_main.attr("__dict__");
glob["std_vector_double"] = boost::python::class_< std::vector<double> >("std_vector_double").def(boost::python::vector_indexing_suite< std::vector<double> >());

boost::python::exec_file("Foo.py", glob, glob);
boost::python::object foo = glob["foo"];
std::vector<double> x;
foo.attr("bar")(x);
// Now x.size() == 1 and x[0] == 3.14

I know this code doesn't work, it's only what I would like to do.

What is the best way to do that ?

My first idea is encapsulate my x as a pointer in another class VectorWrapper, but it looks like an ugly bad idea...

Answer 1

By default, Boost.Python will create a copy, as this is the safest course of action to prevent dangling references. However, one can pass a reference to a C++ object to Python while maintaining ownership in C++, by using boost::python::ptr() or boost::ref(). The C++ code should guarantee that the C++ object's lifetime is at least as long as the Python object.

foo.attr("bar")(boost::ref(x));
foo.attr("bar")(boost::python::ptr(&x));

The above code will construct a std_vector_double Python object that refers to x. When using ptr(), if the pointer is null, then the resulting Python object will be None.

---

Here is a complete example based on the original question that demonstrates the use of boost::ref() to pass a reference to Python.

#include <cmath>  // std::abs
#include <limits> // std::numeric_limits::epsilon
#include <vector>
#include <boost/python.hpp>
#include <boost/python/suite/indexing/vector_indexing_suite.hpp>

int main()
{
  // Initialize Python.
  setenv("PYTHONPATH", ".", 1);
  Py_Initialize();

  namespace python = boost::python;
  try
  {
    // Create the __main__ module.
    python::object main_module = python::import("__main__");
    python::object main_namespace = main_module.attr("__dict__");

    boost::python::class_<std::vector<double>>("std_vector_double")
      .def(python::vector_indexing_suite<std::vector<double>>())
      ;

    // Run the foo.py file within the main namespace.
    python::exec_file("foo.py", main_namespace, main_namespace);

    std::vector<double> x;
    // Pass a reference (C++ maintains ownership) of 'x' to foo.bar().
    main_namespace["foo"].attr("bar")(boost::ref(x));

    // Verify 'x' was modified.
    assert(x.size() == 1);
    assert(std::abs(x[0] - 3.14) <= std::numeric_limits<double>::epsilon());
  }
  catch (const python::error_already_set&)
  {
    PyErr_Print();
    return 1;
  }

  // Do not call Py_Finalize() with Boost.Python.
}

The contents of foo.py are:

class Foo:
    def bar(self, x):
        x.append(3.14)

foo = Foo()