Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using self-defined Cython code from other Cython code

I am currently trying to optimize my Python program and got started with Cython in order to reduce the function calling overhead and perhaps later on include optimized C-libraries functions.

So I ran into the first problem:

I am using composition in my code to create a larger class. So far I have gotten one of my Python classes converted to Cython (which was difficult enough). Here's the code:

import numpy as np
cimport numpy as np
ctypedef np.float64_t dtype_t
ctypedef np.complex128_t cplxtype_t
ctypedef Py_ssize_t index_t

cdef class bendingForcesClass(object):
    cdef dtype_t bendingRigidity
    cdef np.ndarray matrixPrefactor
    cdef np.ndarray bendingForces

    def __init__(self, dtype_t bendingRigidity, np.ndarray[dtype_t, ndim=2] waveNumbersNorm):
        self.bendingRigidity = bendingRigidity
        self.matrixPrefactor = -self.bendingRigidity * waveNumbersNorm ** 2

    cpdef np.ndarray calculate(self, np.ndarray membraneHeight):
        cdef np.ndarray bendingForces
        bendingForces = self.matrixPrefactor * membraneHeight
        return bendingForces

From my composed Python/Cython class I am calling the class-method calculate, so that in my composed class I have the following (reduced) code:

from bendingForcesClass import bendingForcesClass

cdef class membraneClass(object):
    def  __init__(self, systemSideLength, lowerCutoffLength, bendingRigidity):
        self.bendingForces = bendingForcesClass(bendingRigidity, self.waveNumbers.norm)

    def calculateForces(self, heightR):
        return self.bendingForces.calculate(heightR)

I have found out that cpdef makes the method/functions callable from Python and Cython, which is great and works, as long as I don't try to define the type of self.bendingForces beforehand - which according to the documentation (Early Binding For Speed) is necessary in order to remove the function-calling overhead. I have tried the following, which does not work:

from bendingForcesClass import bendingForcesClass
from bendingForcesClass cimport bendingForcesClass

    cdef class membraneClass(object):
        cdef bendingForcesClass bendingForces

        def  __init__(self, systemSideLength, lowerCutoffLength, bendingRigidity):
            self.bendingForces = bendingForcesClass(bendingRigidity, self.waveNumbers.norm)

        def calculateForces(self, heightR):
            return self.bendingForces.calculate(heightR)

With this I get this error, when trying to build membraneClass.pyx with Cython:

membraneClass.pyx:18:6: 'bendingForcesClass' is not a type identifier
building 'membraneClass' extension

Note that the declarations are in two separate files, which makes this more difficult.

So I how do I get this done? I would be very thankful if someone could give me a pointer, as I can't find any information about this, besides the link given above.

Thanks and best regards!

like image 666
packoman Avatar asked Mar 16 '11 19:03

packoman


People also ask

Can Cython compile all Python code?

Cython is a static compiler for Python and Cython programming languages, it simplifies the job of writing Python C extensions. Cython allows us to compile Python code, the result is dynamic libraries that can be used as python modules too.

Can Cython be slower than Python?

Calling the Cython function is faster than calling a Python function call, it's true. But even 30 nanoseconds is rather slow by the standards of compiled languages: for comparison, a C function called by another C function might take only 3 nanoseconds, or much less if it gets inlined.


2 Answers

Disclaimer: This question is very old and I am not sure the current solution would work for 2011 Cython code.

In order to cimport an extension class (cdef class) from another file you need to provide a .pxd file (also known as a definitions file) declaring all C classes, attributes and methods. See Sharing Extension Types in the documentation for reference.

For your example, you would need a file bendingForcesClass.pxd, which declares the class you want to share, as well as all cimports, module level variables, typedefs, etc.:

bendingForcesClass .pxd
# cimports
cimport numpy as np

# typedefy you want to share
ctypedef np.float64_t dtype_t
ctypedef np.complex128_t cplxtype_t
ctypedef Py_ssize_t index_t

cdef class bendingForcesClass:
    # declare C attributes
    cdef dtype_t bendingRigidity
    cdef np.ndarray matrixPrefactor
    cdef np.ndarray bendingForces

    # declare C functions
    cpdef np.ndarray calculate(self, np.ndarray membraneHeight)

    # note that __init__ is missing, it is not a C (cdef) function

All imports, variables, and attributes that now are declared in the .pxd file can (and have to be) removed from the .pyx file:

bendingForcesClass .pyx
import numpy as np

cdef class bendingForcesClass(object):

    def __init__(self, dtype_t bendingRigidity, np.ndarray[dtype_t, ndim=2] waveNumbersNorm):
        self.bendingRigidity = bendingRigidity
        self.matrixPrefactor = -self.bendingRigidity * waveNumbersNorm ** 2

    cpdef np.ndarray calculate(self, np.ndarray membraneHeight):
        cdef np.ndarray bendingForces
        bendingForces = self.matrixPrefactor * membraneHeight
        return bendingForces

Now your cdef class bendingForcesClass can be cimported from other Cython modules, making it a valid type identifier, which should solve your problem.

like image 192
m00am Avatar answered Sep 21 '22 12:09

m00am


You need to use a declaration ".pxd" file and cimport. (Essentially, cimport happens at compile time, while import happens at run time so Cython can't make use of anything important).

Create "utils.pxd":

cdef class MyClass:
    cdef readonly int field
    cdef void go(self, int i)

"utils.pyx" now reads

cdef class MyClass:
    def __init__(self, field):
    self.field = field

cdef void go(self, int i):
    self.field = i

all declarations which have been in the pyx file go into the .pxd file.

Then in mymodule.pyx

from utils import MyClass
from utils cimport MyClass
# other code follows...

// Extended answer from here: Cython: using imported class in a type declaration

like image 22
Davoud Taghawi-Nejad Avatar answered Sep 17 '22 12:09

Davoud Taghawi-Nejad