Polymorphism and pybind11

Question

I have a strange behaviour with pybind11 when I want to use C++ polymorphism in Python. Here is a simple example of my problem:

import polymorphism as plm

a = plm.mylist()

print(a)

a[0].print()
a[1].print()

The output of this script is

[MyBase, MyDerived]

MyBase

MyBase

but the expected output is

[MyBase, MyDerived]

MyBase

MyDerived

because mylist return a std::vector which contains an instance of a derived class (MyDerived) as a second member. The strange thing is that MyDerived is recognized when I print the list as a whole.

Here is the header file of the C++ code:

/* polymorphism.hpp */

#ifndef POLYMORPHISM_HPP
#define POLYMORPHISM_HPP

#include <vector>

class MyBase
{
  public:
    virtual void print() const;
};


class MyDerived : public MyBase
{
  public:
   virtual void print() const;
};


std::vector<MyBase*> mylist();

#endif

And here is the cpp file:

#include "polymorphism.hpp"

#include <iostream>
#include <pybind11/stl.h>
#include <pybind11/pybind11.h>

void MyBase::print() const
{ std::cout << "MyBase" << std::endl; }

void MyDerived::print() const
{ std::cout << "MyDerived" << std::endl; }

std::vector<MyBase*> mylist()
{
  std::vector<MyBase*> list(2);
  list[0] = new MyBase();
  list[1] = new MyDerived();
  return list;
}

PYBIND11_MODULE(polymorphism, m) 
{
  pybind11::class_<MyBase>(m, "MyBase")
    .def(pybind11::init<>())
    .def("print", &MyBase::print)
    .def("__repr__", [](const MyBase &a) { return "MyBase"; });

  pybind11::class_<MyDerived, MyBase>(m, "MyDerived")
    .def(pybind11::init<>())
    .def("print", &MyDerived::print)
    .def("__repr__", [](const MyDerived &a) { return "MyDerived"; });

  m.def("mylist", &mylist, "return a list");
}

EDIT: more surprisingly, when I remove the binding of "print" for MyDerived, I get the following error message

[MyBase, MyDerived]

MyBase

Traceback (most recent call last):

File "test.py", line 8, in

a[1].print()

AttributeError: 'polymorphism.MyDerived' object has no attribute 'print'

This message seems to mean that MyDerived is well recognized while the wrong version of print is called (if I understand well).

EDIT 2: here is a version using trampoline classes. However, this version leads to the same wrong output.

/* polymorphism.hpp */
#ifndef POLYMORPHISM_HPP
#define POLYMORPHISM_HPP

#include <vector>
#include <pybind11/stl.h>
#include <pybind11/pybind11.h>

class MyBase
{
  public:
    virtual void print() const;
};


class MyDerived : public MyBase
{
  public:
    virtual void print() const;
};


std::vector<MyBase*> mylist();

class PyMyBase : public MyBase 
{
  public:
    using MyBase::MyBase; // Inherit constructors
    void print() const override { PYBIND11_OVERLOAD(void, MyBase, print ); }
};

class PyMyDerived : public MyDerived 
{
  public:
    using MyDerived::MyDerived; // Inherit constructors
    void print() const override { PYBIND11_OVERLOAD(void, MyDerived, print);}
};

#endif

Here is the corresponding cpp file:

/* polymorphism.cpp */
#include "polymorphism.hpp"

#include <iostream>

void MyBase::print() const
{ std::cout << "MyBase" << std::endl; }


void MyDerived::print() const
{ std::cout << "MyDerived" << std::endl; }


std::vector<MyBase*> mylist()
{
  std::vector<MyBase*> list(2);
  list[0] = new MyBase();
  list[1] = new MyDerived();
  return list;
}


PYBIND11_MODULE(polymorphism, m) 
{
   pybind11::class_<MyBase, PyMyBase>(m, "MyBase")
     .def(pybind11::init<>())
     .def("print", &MyBase::print)
     .def("__repr__", [](const MyBase &a) { return "MyBase"; });

   pybind11::class_<MyDerived, PyMyDerived>(m, "MyDerived")
     .def(pybind11::init<>())
     .def("print", &MyDerived::print)
     .def("__repr__", [](const MyDerived &a) { return "MyDerived"; });

   m.def("mylist", &mylist, "return a list");
}

Answer 1

I dont know why, but pybind11 seems to have a problem with the raw pointers in mylist(). The example works correctly if you change the return type to vector<unique_ptr<MyBase>>. The following example compiles to a python module example and produces the expected output.

example.cpp:

#include <pybind11/stl.h>
#include <pybind11/pybind11.h>
#include <iostream>
#include <memory>
#include <vector>

class MyBase {
public:
    virtual void print() const {
        std::cout << "MyBase::print()" << std::endl;
    }
};

class MyDerived : public MyBase {
public:
    virtual void print() const override {
        std::cout << "MyDerived::print()" << std::endl;
    }
};

std::vector<std::unique_ptr<MyBase>> mylist() {
    std::vector<std::unique_ptr<MyBase>> v;
    v.push_back(std::make_unique<MyBase>());
    v.push_back(std::make_unique<MyDerived>());
    return v;
}

PYBIND11_MODULE(example, m) {
    pybind11::class_<MyBase>(m, "MyBase")
        .def(pybind11::init<>())
        .def("print", &MyBase::print)
        .def("__repr__", [](MyBase const&) { return "MyBase"; });

    pybind11::class_<MyDerived>(m, "MyDerived")
        .def(pybind11::init<>())
        .def("print", &MyDerived::print)
        .def("__repr__", [](MyDerived const&) { return "MyDerived"; });

    m.def("mylist", &mylist, "returns a list");
}

python shell:

>>> import example
>>> l = example.mylist()
>>> l[0].print()
MyBase::print()
>>> l[1].print()
MyDerived::print()