ImportError from Cython when linking an external C library

Question

I am trying to replicate the work of Flavian Coelho, linked here. He used Cython and the Gnu Scientific Library (GSL) to get a huge speed-up over Python in generating random numbers. When I import my compiled Cython code in Python (with the command import cgibbs), I get the following error:

  ImportError: dlopen(./cgibbs.so, 2): Symbol not found: _gsl_rng_mt19937
  Referenced from: /Users/wesley/scratch/cython/cgibbs.so
  Expected in: dynamic lookup

You'll notice that the complaint is that the symbol _gsl_rng_mt19937 can't be found. The function i am trying to link to is called gsl_rng_mt19937 (no leading underscore), and that is how it appears in my .pyx file. I think Cython is somehow causing the problem by adding that leading underscore.

In order to make troubleshooting easier, I've stripped the code down and posted it below. My system is: Mac OSX 10.7 (Lion) running Python 2.7.2 (32-bit), gcc-4.0 (which I used to compile the GSL libraries in 32-bit form), GSL 1.15, and Cython v0.15.1.

Here is the content of cgibbs.pyx:

#declaring external GSL functions to be used
cdef extern from "math.h":
   double sqrt(double)

cdef double Sqrt(double n):
   return sqrt(n)

cdef extern from "gsl/gsl_rng.h":
   ctypedef struct gsl_rng_type:
       pass
   ctypedef struct gsl_rng:
       pass
   gsl_rng_type *gsl_rng_mt19937
   gsl_rng *gsl_rng_alloc(gsl_rng_type * T)

cdef extern from "gsl/gsl_randist.h":
   double gamma "gsl_ran_gamma"(gsl_rng * r,double,double)
   double gaussian "gsl_ran_gaussian"(gsl_rng * r,double)


cdef gsl_rng *r = gsl_rng_alloc(gsl_rng_mt19937)

The error goes away if I comment out the last line of my cgibbs.pyx, but then I can't actually use the external library... Any insight you can offer is appreciated. Thanks!

Answer 1

Thanks to @ChuiTey, I discovered that otool and nm are the tools on the Mac that can do what objdump does on Linux. With nm I found that the leading underscores are part of the symbol name in the libgsl.a library.

Once I knew that the linker (in my case, ld) was looking for the right name, it was clear that it just wasn't looking in the right place. And that's how I learned that header files don't know the location of the libraries to which they link (Duh!) I just needed to add the option -lgsl to the command that runs the linker (this requires that libgsl.a is in the directory where your linker looks for libraries - on my machine, that's /usr/local/lib.)

I also had to move libgsl.dylib from /usr/local/lib because it was compiled for a 64-bit platform and I am using 32-bit Python.

There must be an easy way to specify the -lgsl linker option when compiling Cython code with distutils or a makefile; for now I am just running gcc-4.0 twice to compile and then link the module. My workflow to compile a Cython module from the bash command line is:

cython cgibbs.pyx
gcc-4.0 -m32 -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c cgibbs.c -o cgibbs.o
gcc-4.0 -bundle -undefined dynamic_lookup -lgsl -arch i386 -g cgibbs.o -o cgibbs.so

Which produces cgibbs.so, a Cython module that can be imported in Python 2.7.