Mixing cdef and regular python attributes in cdef class

Question

i am learning Cython and now experimenting with it. I tried the basic cdef class sample program and it works perfectly.

Now what i want to do is have a mix of cdef and non cdef mix of attributes in the cdef class type, something like this

cdef class Context:
    cdef public int byteIndex, bitIndex

    def __init__(self, msg = "", msg_len = 0):
        self.msg = msg 
        self.msg_len = msg_len 
        self.byteIndex = 0
        self.bitIndex = 7

but as soon as i instantiate the object i get error

!! AttributeError: 'c_asnbase.Context' object has no attribute 'msg'

Does this mean once you define a python class with cdef all self.* attributes have to be cdef defined?

Answer 1

As @DavidW has pointed out, the problem of cdef-classes is that they have no __dict__. You can add __dict__ to the class-definition, if really desired:

%%cython
cdef class A:
    cdef dict __dict__        # now the usual attribute look-up is possible
    cdef readonly int answer 
    def __init__(self):
        self.answer = 42             #cdef attribute
        self.question = "unknown"    #pure python attribute, possible

And now:

a=A()
print(a.answer)
# 42
print(a.question)
# 'unknown' 
a.question = 'Why?'
print(a.question)
# 'Why?' 
setattr(a, 'new_attr', None)
print(a.new_attr)
# None

Note: setattr(a,'new_attr', None) would be not possible if cdef class A were defined without __dict__, but with cdef public object question instead.

Obviously there are additional cost using __dict__, so probably one would use the predefined attributes whenever the performance matters. One of advantages of cdef-classes is smaller memory-footprint (for example because there is no __dict__-slot). So adding __dict__-slot would negate at least some of advantages - one should ask, whether another design would be a better option - but there are obviously scenarios, where adding __dict__-slot makes sense.

Another way would be to create a subclass of the cdef class and use it rather than the base-class.

---

Once the __dict__ slot is defined, instances of class A have the__dict__-attribute (which is not the case for usual cdef-classes). However, __dict__ doesn't contain cdef-attributes, e.g. answer from the example above (no matter whether they are public or not) - only the normal pure python attributes (e.g. question and new_attr in the example above).

Here for the example above:

# no answer-attribute in __dict__:
a.__dict__
# {'new_attr': None, 'question': 'Why?'} 

---

NB: here is the part in the Cython-documentation about dynamic attributes.

Answer 2

Does this mean once you define a python class with cdef all self.* attributes have to be cdef defined?

Yes. This is stated pretty explicitly in the documentation:

Attributes in cdef classes behave differently from attributes in regular classes:

• All attributes must be pre-declared at compile-time
• ...

You can quite happily store a string by defining the attribute to be of type object:

cdef public object msg

---

Internally, the reason for this is that the cdef class does not have a dictionary, which saves space and makes attribute access faster, but it does mean that it cannot have arbitrary attributes added at runtime. This is reasonably similar to using __slots__ in a normal Python class.