swig/python: object does not support indexing

Question

Given this set of files:

foo.h:

#pragma once

#include <stdio.h>

template <class T0> class Foo {
  public:
    T0 m[3];

    Foo(const T0 &a, const T0 &b, const T0 &c) {
        m[0] = a;
        m[1] = b;
        m[2] = c;
    }
    void info() { printf("%d %d %d\n", m[0], m[1], m[2]); }
    // T0 &operator[](int id) { return ((T0 *)m)[id]; }
};

foo.cpp:

#include "foo.h"

foo.i (Attempt1):

%module foo

%{
#include "foo.h"
%}

%include "foo.h"

%template(intFoo) Foo<int>;

%extend Foo{
    T0& __getitem__(int id) { return ((T0 *)m)[id]; }
}

setup.py:

import os
import sys
from setuptools import setup, Extension

foo_module = Extension('_foo',
                           sources=[
                               'foo.i',
                               'foo.cpp'
                           ],
                           swig_opts=['-c++', '-py3', '-builtin'],
                           include_dirs=['.']
                           )

setup(name='foo',
      version='0.1',
      platforms=['Windows', 'Linux'],
      ext_modules=[foo_module],
      py_modules=["foo"],
      )

test.py:

from foo import intFoo

a = intFoo(10,20,30)
print(dir(a))
a.info()
print(a[2])

I build the extension running:

python setup.py build_ext --force -i

but when i try to run test.py i'll get:

TypeError: 'foo.intFoo' object does not support indexing

The statement extend in foo.i is the answer suggested on any other SO related threads, that means I'm using it incorrectly here. Could anyone explain how to fix this so when i run test.py is able to use the [] operator succesfully?

Another attempts:

• Attempt2:

  %module foo
  
  %{
  #include "foo.h"
  %}
  
  %include "foo.h"
  
  %template(intFoo) Foo<int>;
  
  %extend intFoo{
      T0& __getitem__(int id) { return ((T0 *)m)[id]; }
  }
  

  Throws this error TypeError: 'foo.intFoo' object does not support indexing

• Attempt3

  %module foo
  
  %{
  #include "foo.h"
  %}
  
  %include "foo.h"
  
  %extend Foo{
      T0& __getitem__(int id) { return ((T0 *)m)[id]; }
  }
  
  %template(intFoo) Foo<int>;
  

  Throws this error foo_wrap.cpp(3808): error C2065: 'm': undeclared identifier

Answer 1

(Throughout this example I'm working with your first version of foo.i)

First up the %extend you've got needs to be specified before the %template directive to have any effect.

Once we fix that we now get a compiler error from your %extend code:

foo_wrap.cpp: In function 'int& Foo_Sl_int_Sg____getitem__(Foo<int>*, int)':
foo_wrap.cpp:3705:85: error: 'm' was not declared in this scope

This happens because the methods you add with %extend aren't really members of the class you're adding them to. To access m in this context we need to refer to it with $self->m instead. SWIG will replace $self with the appropriate variable for us. (It's worth a quick peek at the generated code to understand how this works)

One useful tip when debugging typemaps or extensions is to search for the code you wrote in the generated output from SWIG - if it's not there then it isn't being applied how you thought.

So once we fix the error about m not being declared we have another problem because you've compiled with -builtin:

In [1]: import foo

In [2]: f=foo.intFoo(1,2,3)

In [3]: f[0]
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-e71eec16918d> in <module>()
----> 1 f[0]

TypeError: 'intFoo' object does not support indexing

In [4]: f.__getitem__(0)
Out[4]: <Swig Object of type 'int *' at 0xa8807d40>

Even though you've added __getitem__, indexing with f[n] still doesn't work. That's happening because with pure C-API classes in Python operator overloading doesn't work the same way. You've added a __getitem__ method successfully, but Python is looking in the builtin type's slots (specifically mp_subscript) for a way to perform the operation. So we need to fix that too. With that done a working foo.i looks like:

%module foo

%{
#include "foo.h"
%}

%feature("python:slot", "mp_subscript", functype="binaryfunc") Foo::__getitem__;

%include "foo.h"

%extend Foo{
    T0& __getitem__(int id) { return ((T0 *)$self->m)[id]; }
}

%template(intFoo) Foo<int>;

So now we can do what you want:

In [1]: import foo

In [2]: f=foo.intFoo(1,2,3)

In [3]: f[0]
Out[3]: <Swig Object of type 'int *' at 0xb4024100>

(You don't actually have to call it __getitem__ any more, because the function gets registered in the slots, it should be possible to call the operator[] without %extend at all for instance)

Finally you probably want to change the return type to const T0&, or write a Python type to proxy objects for non-const int reference better.

Answer 2

I have changed the wrong answer (stating that %extent could only be used without the -builtin option) to something that works, but again only without the 'built-in' option

setup.py

import os
import sys
from setuptools import setup, Extension

foo_module = Extension('_foo',
                           sources=[
                               'foo.i', 'foo.cpp'
                           ],
                           swig_opts=['-c++'],
                           include_dirs=['.']
                           )

setup(name='foo',
      version='0.1',
      platforms=['Windows', 'Linux'],
      ext_modules=[foo_module],
      py_modules=["foo"],
      )

foo.i

%module foo

%{
#include "foo.h"
%}

%include "carrays.i"
%array_functions(int, intArray);

%include "foo.h"

%extend Foo<int> {
%pythoncode %{
def __getitem__(self, id):
  return _foo.intArray_getitem(self.m,id)             
%}
};

%template(intFoo) Foo<int>;

Note, how the extension is generating Python code, which allows you to do many sophisticated things.

Old answer:

According to the SWIG documentation the python layer is stripped off and thereby the %extend feature is ignored (this is incorrect, the proxy objects are not created)

See http://www.swig.org/Doc3.0/SWIGDocumentation.html

36.4.2 Built-in Types

When -builtin is used, the pure python layer is stripped off. Each wrapped class is turned into a new python built-in type which inherits from SwigPyObject, and SwigPyObject instances are returned directly from the wrapped methods. For more information about python built-in extensions, please refer to the python documentation: