ctypes reimplementation of rshift for c_ulong

Question

i am accessing a C library via ctypes and i am stuck with the following problem:

I am generating a "wrapper" (ctypes commands to access the library with ctypes) using ctypeslib. The C library contains macros which are converted to python functions in this step. (To be independent of the libraries internals as much as possible i want to use some of these macros in python.)

One of these macros looks like this:

# using the ctypes types
myuint16_t = c_ushort
myuint32_t = c_ulong

def mymacro(x): return (myuint16_t)((myuint32_t)(x) >> 16) # macro

I want to use the generated function in a seperate module in the following way (inside a function):

return wrapper.mymacro(valueToBeConverted) # valueToBeConverted is an int

But using this line i got the following error:

....   
def mymacro(x): return (myuint16_t)((myuint32_t)(x) >> 16) # macro
TypeError: unsupported operand type(s) for >>: 'c_ulong' and 'int'

(I know that the common way to shift a c_ulong is c_ulongvar.value >> x but i would have to patch the generated wrapper every time something changes in the C library. So i try to avoid this).

It seems that the __rshift__ implementation of c_ulong can not be used here.

print c_ulong.__rshift__
# throws AttributeError: type object 'c_ulong' has no attribute '__rshift__'

Hm, seems strange... So i decided to reimplement the __rshift__ method of c_ulong to get it working:

from ctypes import *
from types import MethodType

def rshift(self, val):
    print self.value >> val

# create an unbound method which applies to all (even existing) instances
c_ulong.__rshift__ = MethodType(rshift, None, c_ulong)

a = c_ulong(1)
a >> 16

But it does not fix the problem. I am still getting an error:

a >> 16
TypeError: unsupported operand type(s) for >>: 'c_ulong' and 'int'

Is it possible that the __rshift__ method can be only used for two instances of the same class? I tried the following:

def rshift(self, val):
    print self.value >> int(val.value)

a = c_ulong(1)
a >> c_ulong(16) 

and it works. But it would also mean i would still have to patch the generated wrapper.

So: Does anybody know whats the trick here?

UPDATE:

The solution of @eryksun worked. I am using:

from ctypes import *
# from types import MethodType

def _rshift(self, other):
    if hasattr(other, 'value'):
        other = other.value
    return c_ulong(self.value >> other)

def _lshift(self, other):
    if hasattr(other, 'value'):
        other = other.value
    return c_ulong(self.value << other)

def _coerce(self, other):
    try:
        return self, self.__class__(other)
    except TypeError:
        return NotImplemented

# Add the functions to the type. A method is created when
# accessed as an attribute of an instance.
c_ulong.__lshift__ = _lshift
c_ulong.__rshift__ = _rshift
c_ulong.__coerce__ = _coerce

Answer 1

Since the _ctypes._SimpleCData type doesn't have the Py_TPFLAGS_CHECKTYPES flag, 2.x subclasses are treated as old-style numbers that use __coerce__ in binary operations. See Objects/abstract.c for the calling scheme and the implementation in the function binary_op1.

For demonstration purposes this flag can be toggled on the type object, which you only need to define (vaguely with a lot of void *) up to the tp_flags field.

Hacking the PyTypeObject

from ctypes import *
import _ctypes

Py_TPFLAGS_CHECKTYPES = 1 << 4

class PyTypeObject(Structure):
    _fields_ = (('ob_refcnt', c_ssize_t),
                ('ob_type', c_void_p),
                ('ob_size', c_ssize_t),
                ('tp_name', c_char_p),
                ('tp_basicsize', c_ssize_t),
                ('tp_itemsize', c_ssize_t),
                ('tp_dealloc', c_void_p),
                ('tp_print', c_void_p),
                ('tp_getattr', c_void_p),
                ('tp_setattr', c_void_p),
                ('tp_compare', c_void_p),
                ('tp_repr', c_void_p),
                ('tp_as_number', c_void_p),
                ('tp_as_sequence', c_void_p),
                ('tp_as_mapping', c_void_p),
                ('tp_hash', c_void_p),
                ('tp_call', c_void_p),
                ('tp_str', c_void_p),
                ('tp_getattro', c_void_p),
                ('tp_setattro', c_void_p),
                ('tp_as_buffer', c_void_p),
                ('tp_flags', c_long))

Next, create an unsigned long subclass, and use the from_address factory to create a PyTypeObject for it. Get the address with built-in id, which is an implementation detail specific to CPython:

class c_ulong(_ctypes._SimpleCData):
    _type_ = "L"

    def __rshift__(self, other):
        print '__rshift__', self, other
        if hasattr(other, 'value'):
            other = other.value
        return c_ulong(self.value >> other)

c_ulong_type = PyTypeObject.from_address(id(c_ulong))

Demo

>>> a = c_ulong(16)
>>> b = c_ulong(2)

>>> a >> b
__rshift__ c_ulong(16L) c_ulong(2L)
c_ulong(4L)

>>> a >> 2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for >>: 'c_ulong' and 'int'

The last step failed as expected. Now set the flag:

>>> c_ulong_type.tp_flags |= Py_TPFLAGS_CHECKTYPES

>>> a >> 2
__rshift__ c_ulong(16L) 2
c_ulong(4L)

Problem solved? But that's a hack. Try again with __coerce__ implemented.

---

Implement __coerce__

class c_ulong(_ctypes._SimpleCData):
    _type_ = "L"

    def __rshift__(self, other):
        print '__rshift__', self, other
        if hasattr(other, 'value'):
            other = other.value
        return c_ulong(self.value >> other)

    def __coerce__(self, other):
        print '__coerce__', self, other
        try:
            return self, self.__class__(other)
        except TypeError:
            return NotImplemented

Demo

>>> a = c_ulong(16)
>>> b = c_ulong(2)

>>> a >> 2
__coerce__ c_ulong(16L) 2
__rshift__ c_ulong(16L) c_ulong(2L)
c_ulong(4L)

>>> 16 >> b
__coerce__ c_ulong(2L) 16
__rshift__ c_ulong(16L) c_ulong(2L)
c_ulong(4L)

Of course it fails if a c_ulong can't be created, such as for a float:

>>> a >> 2.0
__coerce__ c_ulong(16L) 2.0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for >>: 'c_ulong' and 'float'