Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python binding for C++ operator overloading

I have a class similar to the following:

class A {
    vector<double> v;
    double& x(int i) { return v[2*i]; }
    double& y(int i) { return v[2*i+1]; }
    double x(int i) const { return v[2*i]; }
    double y(int i) const { return v[2*i+1]; }
}

I want to have the following Python code work:

a = A()
a.x[0] = 4
print a.x[0]

I was thinking of __setattr__ and __getattr__, but not sure if it works. An alternative is to implement the following Python:

a = A()
a['x', 0] = 4
print a['x', 0]

not as good as the previous one, but might be easier to implement (with __slice__ ?).

PS. I am using sip to do the binding.

Thanks.

like image 517
Forest Yang Avatar asked Jul 12 '12 15:07

Forest Yang


People also ask

Is Python support for operator overloading?

Python does not limit operator overloading to arithmetic operators only. We can overload comparison operators as well.

What are Python C bindings?

"Bindings" are implemented either as a pure Python library using ctypes or as a dynamic-link library using Python/C API. The second option is sometimes used with tools like SWIG which make the task easier by taking care of generating the "boiler-plate" code or Boost.

What is the most popular form of operator overloading in Python?

A very popular and convenient example is the Addition (+) operator. Just think how the '+' operator operates on two numbers and the same operator operates on two strings. It performs “Addition” on numbers whereas it performs “Concatenation” on strings.


1 Answers

It is possible with __getattr__ and custom %MethodCode; however, there are a few points to take into consideration:

  • An intermediate type/object needs to be created, as a.x will return an object that provides __getitem__ and __setitem__. Both methods should raise an IndexError when out of bounds occurs, as this is part of the old protocol used to iterate via __getitem__; without it, a crash would occur when iterating over a.x.
  • In order to guarantee the lifetime of the vector, the a.x object needs to maintain a reference to the object that owns the vector (a). Consider the following code:

    a = A()
    x = a.x
    a = None # If 'x' has a reference to 'a.v' and not 'a', then it may have a
             # dangling reference, as 'a' is refcounted by python, and 'a.v' is
             # not refcounted.
    
  • Writing %MethodCode can be difficult, especially when having to manage the reference counting during error cases. It requires an understanding of the python C API and SIP.

For an alternative solution, consider:

  • Design the python bindings to provide functionality.
  • Design class(es) in python to provide the pythonic interface that uses the bindings.

While the approach has a few drawbacks, such as the code is separated into more files that may need to be distributed with the library, it does provide some major benefits:

  • It is much easier to implement a pythonic interface in python than in C or the interoperability library's interface.
  • Support for slicing, iterators, etc. can be more naturally implemented in python, instead of having to manage it through the C API.
  • Can leverage python's garbage collector to manage the lifetime of the underlying memory.
  • The pythonic interface is decoupled from whatever implementation is being used to provide interoperability between python and C++. With a flatter and simpler binding interface, changing between implementations, such as Boost.Python and SIP, is much easier.

Here is an walk-through demonstrating this approach. First, we start with the basic A class. In this example, I have provided a constructor that will set some initial data.

a.hpp:

#ifndef A_HPP
#define A_HPP

#include <vector>

class A
{
  std::vector< double > v;
public:
  A() { for ( int i = 0; i < 6; ++i ) v.push_back( i ); }
  double& x( int i )         { return v[2*i];       }
  double  x( int i ) const   { return v[2*i];       }
  double& y( int i )         { return v[2*i+1];     }
  double  y( int i ) const   { return v[2*i+1];     }
  std::size_t size() const   { return v.size() / 2; }
};

#endif  // A_HPP

Before doing the bindings, lets examine the A interface. While it is an easy interface to use in C++, it has some difficulties in python:

  • Python does not support overloaded methods, and idioms to support overloading will fail when the argument type/counts are the same.
  • The concept of a reference to a double (float in Python) is different between the two languages. In Python, the float is an immutable type, so its value cannot be changed. For example, in Python the statement n = a.x[0] binds n to reference the float object returned from a.x[0]. The assignment n = 4 rebinds n to reference the int(4) object; it does not set a.x[0] to 4.
  • __len__ expects int, not std::size_t.

Lets create a basic intermediate class that will help simplify the bindings.

pya.hpp:

#ifndef PYA_HPP
#define PYA_HPP

#include "a.hpp"

struct PyA: A
{
  double get_x( int i )           { return x( i ); }
  void   set_x( int i, double v ) { x( i ) = v;    }
  double get_y( int i )           { return y( i ); }
  void   set_y( int i, double v ) { y( i ) = v;    }
  int    length()                 { return size(); }
};

#endif // PYA_HPP

Great! PyA now provides member functions that do not return references, and length is returned as an int. It is not the best of interfaces, the bindings are being designed to provide the needed functionality, rather than the desired interface.

Now, lets write some simple bindings that will create class A in the cexample module.

Here is the bindings in SIP:

%Module cexample

class PyA /PyName=A/
{
%TypeHeaderCode
#include "pya.hpp"
%End
public:
  double get_x( int );
  void set_x( int, double );
  double get_y( int );
  void set_y( int, double );
  int __len__();
  %MethodCode
    sipRes = sipCpp->length();
  %End
};

Or if you prefer Boost.Python:

#include "pya.hpp"
#include <boost/python.hpp>

BOOST_PYTHON_MODULE(cexample)
{
  using namespace boost::python;
  class_< PyA >( "A" )
    .def( "get_x",   &PyA::get_x  )
    .def( "set_x",   &PyA::set_x  )
    .def( "get_y",   &PyA::get_y  )
    .def( "set_y",   &PyA::set_y  )
    .def( "__len__", &PyA::length )
    ;
}

Due to the PyA intermediate class, both of the bindings are fairly simple. Additionally, this approach requires less SIP and Python C API knowledge, as it requires less code within %MethodCode blocks.

Finally, create example.py that will provide the desired pythonic interface:

class A:
    class __Helper:
        def __init__( self, data, getter, setter ):
            self.__data   = data
            self.__getter = getter
            self.__setter = setter

        def __getitem__( self, index ):
            if len( self ) <= index:
                raise IndexError( "index out of range" )
            return self.__getter( index )

        def __setitem__( self, index, value ):
            if len( self ) <= index:
                raise IndexError( "index out of range" )
            self.__setter( index, value )

        def __len__( self ):
            return len( self.__data )

    def __init__( self ):
        import cexample
        a = cexample.A()
        self.x = A.__Helper( a, a.get_x, a.set_x )
        self.y = A.__Helper( a, a.get_y, a.set_y )

In the end, the bindings provide the functionality we need, and python creates the interface we want. It is possible to have the bindings provide the interface; however, this can require a rich understanding of the differences between the two languages and the binding implementation.

>>> from example import A
>>> a = A()
>>> for x in a.x:
...   print x
... 
0.0
2.0
4.0
>>> a.x[0] = 4
>>> for x in a.x:
...   print x
... 
4.0
2.0
4.0
>>> x = a.x
>>> a = None
>>> print x[0]
4.0
like image 192
Tanner Sansbury Avatar answered Sep 20 '22 12:09

Tanner Sansbury