Python binding for C++ operator overloading

Question

I have a class similar to the following:

class A {
    vector<double> v;
    double& x(int i) { return v[2*i]; }
    double& y(int i) { return v[2*i+1]; }
    double x(int i) const { return v[2*i]; }
    double y(int i) const { return v[2*i+1]; }
}

I want to have the following Python code work:

a = A()
a.x[0] = 4
print a.x[0]

I was thinking of __setattr__ and __getattr__, but not sure if it works. An alternative is to implement the following Python:

a = A()
a['x', 0] = 4
print a['x', 0]

not as good as the previous one, but might be easier to implement (with __slice__ ?).

PS. I am using sip to do the binding.

Thanks.

Answer 1

It is possible with __getattr__ and custom %MethodCode; however, there are a few points to take into consideration:

• An intermediate type/object needs to be created, as a.x will return an object that provides __getitem__ and __setitem__. Both methods should raise an IndexError when out of bounds occurs, as this is part of the old protocol used to iterate via __getitem__; without it, a crash would occur when iterating over a.x.

• In order to guarantee the lifetime of the vector, the a.x object needs to maintain a reference to the object that owns the vector (a). Consider the following code:

  a = A()
  x = a.x
  a = None # If 'x' has a reference to 'a.v' and not 'a', then it may have a
           # dangling reference, as 'a' is refcounted by python, and 'a.v' is
           # not refcounted.
  

• Writing %MethodCode can be difficult, especially when having to manage the reference counting during error cases. It requires an understanding of the python C API and SIP.

For an alternative solution, consider:

• Design the python bindings to provide functionality.
• Design class(es) in python to provide the pythonic interface that uses the bindings.

While the approach has a few drawbacks, such as the code is separated into more files that may need to be distributed with the library, it does provide some major benefits:

• It is much easier to implement a pythonic interface in python than in C or the interoperability library's interface.
• Support for slicing, iterators, etc. can be more naturally implemented in python, instead of having to manage it through the C API.
• Can leverage python's garbage collector to manage the lifetime of the underlying memory.
• The pythonic interface is decoupled from whatever implementation is being used to provide interoperability between python and C++. With a flatter and simpler binding interface, changing between implementations, such as Boost.Python and SIP, is much easier.

---

Here is an walk-through demonstrating this approach. First, we start with the basic A class. In this example, I have provided a constructor that will set some initial data.

a.hpp:

#ifndef A_HPP
#define A_HPP

#include <vector>

class A
{
  std::vector< double > v;
public:
  A() { for ( int i = 0; i < 6; ++i ) v.push_back( i ); }
  double& x( int i )         { return v[2*i];       }
  double  x( int i ) const   { return v[2*i];       }
  double& y( int i )         { return v[2*i+1];     }
  double  y( int i ) const   { return v[2*i+1];     }
  std::size_t size() const   { return v.size() / 2; }
};

#endif  // A_HPP

Before doing the bindings, lets examine the A interface. While it is an easy interface to use in C++, it has some difficulties in python:

• Python does not support overloaded methods, and idioms to support overloading will fail when the argument type/counts are the same.
• The concept of a reference to a double (float in Python) is different between the two languages. In Python, the float is an immutable type, so its value cannot be changed. For example, in Python the statement n = a.x[0] binds n to reference the float object returned from a.x[0]. The assignment n = 4 rebinds n to reference the int(4) object; it does not set a.x[0] to 4.
• __len__ expects int, not std::size_t.

Lets create a basic intermediate class that will help simplify the bindings.

pya.hpp:

#ifndef PYA_HPP
#define PYA_HPP

#include "a.hpp"

struct PyA: A
{
  double get_x( int i )           { return x( i ); }
  void   set_x( int i, double v ) { x( i ) = v;    }
  double get_y( int i )           { return y( i ); }
  void   set_y( int i, double v ) { y( i ) = v;    }
  int    length()                 { return size(); }
};

#endif // PYA_HPP

Great! PyA now provides member functions that do not return references, and length is returned as an int. It is not the best of interfaces, the bindings are being designed to provide the needed functionality, rather than the desired interface.

Now, lets write some simple bindings that will create class A in the cexample module.

Here is the bindings in SIP:

%Module cexample

class PyA /PyName=A/
{
%TypeHeaderCode
#include "pya.hpp"
%End
public:
  double get_x( int );
  void set_x( int, double );
  double get_y( int );
  void set_y( int, double );
  int __len__();
  %MethodCode
    sipRes = sipCpp->length();
  %End
};

Or if you prefer Boost.Python:

#include "pya.hpp"
#include <boost/python.hpp>

BOOST_PYTHON_MODULE(cexample)
{
  using namespace boost::python;
  class_< PyA >( "A" )
    .def( "get_x",   &PyA::get_x  )
    .def( "set_x",   &PyA::set_x  )
    .def( "get_y",   &PyA::get_y  )
    .def( "set_y",   &PyA::set_y  )
    .def( "__len__", &PyA::length )
    ;
}

Due to the PyA intermediate class, both of the bindings are fairly simple. Additionally, this approach requires less SIP and Python C API knowledge, as it requires less code within %MethodCode blocks.

Finally, create example.py that will provide the desired pythonic interface:

class A:
    class __Helper:
        def __init__( self, data, getter, setter ):
            self.__data   = data
            self.__getter = getter
            self.__setter = setter

        def __getitem__( self, index ):
            if len( self ) <= index:
                raise IndexError( "index out of range" )
            return self.__getter( index )

        def __setitem__( self, index, value ):
            if len( self ) <= index:
                raise IndexError( "index out of range" )
            self.__setter( index, value )

        def __len__( self ):
            return len( self.__data )

    def __init__( self ):
        import cexample
        a = cexample.A()
        self.x = A.__Helper( a, a.get_x, a.set_x )
        self.y = A.__Helper( a, a.get_y, a.set_y )

In the end, the bindings provide the functionality we need, and python creates the interface we want. It is possible to have the bindings provide the interface; however, this can require a rich understanding of the differences between the two languages and the binding implementation.

>>> from example import A
>>> a = A()
>>> for x in a.x:
...   print x
... 
0.0
2.0
4.0
>>> a.x[0] = 4
>>> for x in a.x:
...   print x
... 
4.0
2.0
4.0
>>> x = a.x
>>> a = None
>>> print x[0]
4.0