Simple wrapping of C code with cython

Question

I have a number of C functions, and I would like to call them from python. cython seems to be the way to go, but I can't really find an example of how exactly this is done. My C function looks like this:

void calculate_daily ( char *db_name, int grid_id, int year,                        double *dtmp, double *dtmn, double *dtmx,                         double *dprec, double *ddtr, double *dayl,                         double *dpet, double *dpar ) ; 

All I want to do is to specify the first three parameters (a string and two integers), and recover 8 numpy arrays (or python lists. All the double arrays have N elements). My code assumes that the pointers are pointing to an already allocated chunk of memory. Also, the produced C code ought to link to some external libraries.

Answer 1

Here's a tiny but complete example of passing numpy arrays to an external C function, logically

fc( int N, double* a, double* b, double* z )  # z = a + b 

using Cython. (This is surely well-known to those who know it well. Comments are welcome. Last change: 23 Feb 2011, for Cython 0.14.)

First read or skim Cython build and Cython with NumPy .

2 steps:

• python f-setup.py build_ext --inplace
  turns f.pyx and fc.cpp -> f.so, a dynamic library
• python test-f.py
import f loads f.so; f.fpy( ... ) calls the C fc( ... ).

python f-setup.py uses distutils to run cython, compile and link:
cython f.pyx -> f.cpp
compile f.cpp and fc.cpp
link f.o fc.o -> f.so, a dynamic lib that python import f will load.

For students, I'd suggest: make a diagram of these steps, look through the files below, then download and run them.

(distutils is a huge, convoluted package used to make Python packages for distribution, and install them. Here we're using just a small part of it to compile and link to f.so. This step has nothing to do with Cython, but it can be confusing; simple mistakes in a .pyx can cause pages of obscure error messages from g++ compile and link. See also distutils doc and/or SO questions on distutils .)

Like make, setup.py will rerun cython f.pyx and g++ -c ... f.cpp if f.pyx is newer than f.cpp.
To cleanup, rm -r build/ .

An alternative to setup.py would be to run the steps separately, in a script or Makefile:
cython --cplus f.pyx -> f.cpp # see cython -h
g++ -c ... f.cpp -> f.o
g++ -c ... fc.cpp -> fc.o
cc-lib f.o fc.o -> dynamic library f.so.
Modify the cc-lib-mac wrapper below for your platform and installation: it's not pretty, but small.

For real examples of Cython wrapping C, look at .pyx files in just about any SciKit .

See also: Cython for NumPy users and SO questions/tagged/cython .

---

To unpack the following files, cut-paste the lot to one big file, say cython-numpy-c-demo, then in Unix (in a clean new directory) run sh cython-numpy-c-demo.

#-------------------------------------------------------------------------------- cat >f.pyx <<\! # f.pyx: numpy arrays -> extern from "fc.h" # 3 steps: # cython f.pyx  -> f.c # link: python f-setup.py build_ext --inplace  -> f.so, a dynamic library # py test-f.py: import f gets f.so, f.fpy below calls fc()  import numpy as np cimport numpy as np  cdef extern from "fc.h":      int fc( int N, double* a, double* b, double* z )  # z = a + b  def fpy( N,     np.ndarray[np.double_t,ndim=1] A,     np.ndarray[np.double_t,ndim=1] B,     np.ndarray[np.double_t,ndim=1] Z ):     """ wrap np arrays to fc( a.data ... ) """     assert N <= len(A) == len(B) == len(Z)     fcret = fc( N, <double*> A.data, <double*> B.data, <double*> Z.data )         # fcret = fc( N, A.data, B.data, Z.data )  grr char*     return fcret  !  #-------------------------------------------------------------------------------- cat >fc.h <<\! // fc.h: numpy arrays from cython , double*  int fc( int N, const double a[], const double b[], double z[] ); !  #-------------------------------------------------------------------------------- cat >fc.cpp <<\! // fc.cpp: z = a + b, numpy arrays from cython  #include "fc.h" #include <stdio.h>  int fc( int N, const double a[], const double b[], double z[] ) {     printf( "fc: N=%d a[0]=%f b[0]=%f \n", N, a[0], b[0] );     for( int j = 0;  j < N;  j ++ ){         z[j] = a[j] + b[j];     }     return N; } !  #-------------------------------------------------------------------------------- cat >f-setup.py <<\! # python f-setup.py build_ext --inplace #   cython f.pyx -> f.cpp #   g++ -c f.cpp -> f.o #   g++ -c fc.cpp -> fc.o #   link f.o fc.o -> f.so  # distutils uses the Makefile distutils.sysconfig.get_makefile_filename() # for compiling and linking: a sea of options.  # http://docs.python.org/distutils/introduction.html # http://docs.python.org/distutils/apiref.html  20 pages ... # https://stackoverflow.com/questions/tagged/distutils+python  import numpy from distutils.core import setup from distutils.extension import Extension from Cython.Distutils import build_ext # from Cython.Build import cythonize  ext_modules = [Extension(     name="f",     sources=["f.pyx", "fc.cpp"],         # extra_objects=["fc.o"],  # if you compile fc.cpp separately     include_dirs = [numpy.get_include()],  # .../site-packages/numpy/core/include     language="c++",         # libraries=         # extra_compile_args = "...".split(),         # extra_link_args = "...".split()     )]  setup(     name = 'f',     cmdclass = {'build_ext': build_ext},     ext_modules = ext_modules,         # ext_modules = cythonize(ext_modules)  ? not in 0.14.1     # version=     # description=     # author=     # author_email=     )  # test: import f !  #-------------------------------------------------------------------------------- cat >test-f.py <<\! #!/usr/bin/env python # test-f.py  import numpy as np import f  # loads f.so from cc-lib: f.pyx -> f.c + fc.o -> f.so  N = 3 a = np.arange( N, dtype=np.float64 ) b = np.arange( N, dtype=np.float64 ) z = np.ones( N, dtype=np.float64 ) * np.NaN  fret = f.fpy( N, a, b, z ) print "fpy -> fc z:", z  !  #-------------------------------------------------------------------------------- cat >cc-lib-mac <<\! #!/bin/sh me=${0##*/} case $1 in "" )     set --  f.cpp fc.cpp ;;  # default: g++ these -h* | --h* )     echo " $me [g++ flags] xx.c yy.cpp zz.o ...     compiles .c .cpp .o files to a dynamic lib xx.so "     exit 1 esac  # Logically this is simple, compile and link, # but platform-dependent, layers upon layers, gloom, doom  base=${1%.c*} base=${base%.o} set -x  g++ -dynamic -arch ppc \     -bundle -undefined dynamic_lookup \     -fno-strict-aliasing -fPIC -fno-common -DNDEBUG `# -g` -fwrapv \     -isysroot /Developer/SDKs/MacOSX10.4u.sdk \     -I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 \     -I${Pysite?}/numpy/core/include \     -O2 -Wall \     "$@" \     -o $base.so  # undefs: nm -gpv $base.so | egrep '^ *U _+[^P]' !  # 23 Feb 2011 13:38 

Answer 2

The following Cython code from http://article.gmane.org/gmane.comp.python.cython.user/5625 doesn't require explicit casts and also handles non-continous arrays:

def fpy(A):     cdef np.ndarray[np.double_t, ndim=2, mode="c"] A_c     A_c = np.ascontiguousarray(A, dtype=np.double)     fc(&A_c[0,0])