pybind how can I operate over a py::list object

Question

For a better understanding of how to pass arguments from Python to C++ functions with the pybind library, I wanted to build a small dummy/demo code where I could receive a Python list on the C++ side, cast it to a float pointer object, and then print it.

Though I know I can use the py::list class I haven't figured out the methods available of this class. I looked in the documentation reference, and then in the code (list.h, stl.h) and couldn't figure out which methods where available.

What is the equivalent of __getitem__? Do I have every python method available for py::list?

Answer 1

The code you are looking for is here:

class list : public object {
public:
    PYBIND11_OBJECT_CVT(list, object, PyList_Check, PySequence_List)
    explicit list(size_t size = 0) : object(PyList_New((ssize_t) size), stolen_t{}) {
        if (!m_ptr) pybind11_fail("Could not allocate list object!");
    }
    size_t size() const { return (size_t) PyList_Size(m_ptr); }
    detail::list_accessor operator[](size_t index) const { return {*this, index}; }
    detail::list_iterator begin() const { return {*this, 0}; }
    detail::list_iterator end() const { return {*this, PyList_GET_SIZE(m_ptr)}; }
    template <typename T> void append(T &&val) const {
        PyList_Append(m_ptr, detail::object_or_cast(std::forward<T>(val)).ptr());
    }
};

Also keep in mind that py::list inherits from py::object, which in turn inherits from py::handle (this also means that you are passing by reference). In my experience, there is very little documentation for this kind of usage, and reading the code is your best bet.

We can see from the class definition that we can use the member functions size, operator[], begin, end (C++ iterators!) and append (templated!). If this is not enough, you can use attr to access python attributes (including methods). Here is an example:

Python code (some_python.py):

import cppimport
cpp = cppimport.imp("some_cpp")

l = [1,2,3,4,5]
cpp.test(l)
print('after C++', l)

cpp.float_cast(l)

C++ code (some_cpp.cpp):

/* <%
setup_pybind11(cfg)
%> */

#include <pybind11/pybind11.h>
#include <iostream>
#include <string>

namespace py = pybind11;

void test(py::list l) {
    l.attr("pop")();
    std::cout << "List has length " << l.size() << std::endl;
    for (py::handle obj : l) {  // iterators!
        std::cout << "  - " << obj.attr("__str__")().cast<std::string>() << std::endl;
    }
    l.append(10);  // automatic casting (through templating)!
}

void float_cast(py::list l) {
    float f = l.cast<float>();
}

PYBIND11_MODULE(some_cpp, m) {
    m.def("test", &test);
    m.def("float_cast", &float_cast);
}

Output:

List has length 4
  - 1
  - 2
  - 3
  - 4
after C++ [1, 2, 3, 4, 10]
Traceback (most recent call last):
  File "some_python.py", line 9, in <module>
    cpp.float_cast(l)
RuntimeError: Unable to cast Python instance to C++ type (compile in debug mode for details)

As you can see I've also included your specific question of casting to a float. Here I've used the cast method of py::handle, which gives a nice exception. You may try to "directly" cast the object (something like float* f = (float*) &l;) but this will give you garbage and I guess this is not what you're looking for.

One more remark: pybind/stl.h enables conversion between Python's standard types and C++ versions. For example, a list may be converted to a std::vector<int>, including typechecks. An important effect of this is that the data is passed as a copy instead of as a reference.