Using self-defined Cython code from other Cython code

Question

I am currently trying to optimize my Python program and got started with Cython in order to reduce the function calling overhead and perhaps later on include optimized C-libraries functions.

So I ran into the first problem:

I am using composition in my code to create a larger class. So far I have gotten one of my Python classes converted to Cython (which was difficult enough). Here's the code:

import numpy as np
cimport numpy as np
ctypedef np.float64_t dtype_t
ctypedef np.complex128_t cplxtype_t
ctypedef Py_ssize_t index_t

cdef class bendingForcesClass(object):
    cdef dtype_t bendingRigidity
    cdef np.ndarray matrixPrefactor
    cdef np.ndarray bendingForces

    def __init__(self, dtype_t bendingRigidity, np.ndarray[dtype_t, ndim=2] waveNumbersNorm):
        self.bendingRigidity = bendingRigidity
        self.matrixPrefactor = -self.bendingRigidity * waveNumbersNorm ** 2

    cpdef np.ndarray calculate(self, np.ndarray membraneHeight):
        cdef np.ndarray bendingForces
        bendingForces = self.matrixPrefactor * membraneHeight
        return bendingForces

From my composed Python/Cython class I am calling the class-method calculate, so that in my composed class I have the following (reduced) code:

from bendingForcesClass import bendingForcesClass

cdef class membraneClass(object):
    def  __init__(self, systemSideLength, lowerCutoffLength, bendingRigidity):
        self.bendingForces = bendingForcesClass(bendingRigidity, self.waveNumbers.norm)

    def calculateForces(self, heightR):
        return self.bendingForces.calculate(heightR)

I have found out that cpdef makes the method/functions callable from Python and Cython, which is great and works, as long as I don't try to define the type of self.bendingForces beforehand - which according to the documentation (Early Binding For Speed) is necessary in order to remove the function-calling overhead. I have tried the following, which does not work:

from bendingForcesClass import bendingForcesClass
from bendingForcesClass cimport bendingForcesClass

    cdef class membraneClass(object):
        cdef bendingForcesClass bendingForces

        def  __init__(self, systemSideLength, lowerCutoffLength, bendingRigidity):
            self.bendingForces = bendingForcesClass(bendingRigidity, self.waveNumbers.norm)

        def calculateForces(self, heightR):
            return self.bendingForces.calculate(heightR)

With this I get this error, when trying to build membraneClass.pyx with Cython:

membraneClass.pyx:18:6: 'bendingForcesClass' is not a type identifier
building 'membraneClass' extension

Note that the declarations are in two separate files, which makes this more difficult.

So I how do I get this done? I would be very thankful if someone could give me a pointer, as I can't find any information about this, besides the link given above.

Thanks and best regards!

Answer 1

Disclaimer: This question is very old and I am not sure the current solution would work for 2011 Cython code.

In order to cimport an extension class (cdef class) from another file you need to provide a .pxd file (also known as a definitions file) declaring all C classes, attributes and methods. See Sharing Extension Types in the documentation for reference.

For your example, you would need a file bendingForcesClass.pxd, which declares the class you want to share, as well as all cimports, module level variables, typedefs, etc.:

bendingForcesClass .pxd

# cimports
cimport numpy as np

# typedefy you want to share
ctypedef np.float64_t dtype_t
ctypedef np.complex128_t cplxtype_t
ctypedef Py_ssize_t index_t

cdef class bendingForcesClass:
    # declare C attributes
    cdef dtype_t bendingRigidity
    cdef np.ndarray matrixPrefactor
    cdef np.ndarray bendingForces

    # declare C functions
    cpdef np.ndarray calculate(self, np.ndarray membraneHeight)

    # note that __init__ is missing, it is not a C (cdef) function

All imports, variables, and attributes that now are declared in the .pxd file can (and have to be) removed from the .pyx file:

bendingForcesClass .pyx

import numpy as np

cdef class bendingForcesClass(object):

    def __init__(self, dtype_t bendingRigidity, np.ndarray[dtype_t, ndim=2] waveNumbersNorm):
        self.bendingRigidity = bendingRigidity
        self.matrixPrefactor = -self.bendingRigidity * waveNumbersNorm ** 2

    cpdef np.ndarray calculate(self, np.ndarray membraneHeight):
        cdef np.ndarray bendingForces
        bendingForces = self.matrixPrefactor * membraneHeight
        return bendingForces

Now your cdef class bendingForcesClass can be cimported from other Cython modules, making it a valid type identifier, which should solve your problem.

Answer 2

You need to use a declaration ".pxd" file and cimport. (Essentially, cimport happens at compile time, while import happens at run time so Cython can't make use of anything important).

Create "utils.pxd":

cdef class MyClass:
    cdef readonly int field
    cdef void go(self, int i)

"utils.pyx" now reads

cdef class MyClass:
    def __init__(self, field):
    self.field = field

cdef void go(self, int i):
    self.field = i

all declarations which have been in the pyx file go into the .pxd file.

Then in mymodule.pyx

from utils import MyClass
from utils cimport MyClass
# other code follows...

// Extended answer from here: Cython: using imported class in a type declaration