Cython: Segmentation Fault Using API Embedding Cython to C

Question

I'm trying to embed Cython code into C following O'reilly Cython book chapter 8. I found this paragraph on Cython's documentation but still don't know what should I do:

If the C code wanting to use these functions is part of more than one shared library or executable, then import_modulename() function needs to be called in each of the shared libraries which use these functions. If you crash with a segmentation fault (SIGSEGV on linux) when calling into one of these api calls, this is likely an indication that the shared library which contains the api call which is generating the segmentation fault does not call the import_modulename() function before the api call which crashes.

I'm running Python 3.4, Cython 0.23 and GCC 5 on OS X. The source code are transcendentals.pyx and main.c:

main.c

#include "transcendentals_api.h"
#include <math.h>
#include <stdio.h>

int main(int argc, char **argv)
{
  Py_SetPythonHome(L"/Users/spacegoing/anaconda");
  Py_Initialize();
  import_transcendentals();
  printf("pi**e: %f\n", pow(get_pi(), get_e()));

  Py_Finalize();
    return 0;
}

transcendentals.pyx

cdef api double get_pi():
    return 3.1415926

cdef api double get_e():
    print("calling get_e()")
    return 2.718281828

I'm compiling those files using setup.py and Makefile:

setup.py:

from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize

setup(
    ext_modules=cythonize([
        Extension("transcendentals", ["transcendentals.pyx"])
    ])
)

Makefile

python-config=/Users/spacegoing/anaconda/bin/python3-config
ldflags:=$(shell $(python-config) --ldflags)
cflags:=$(shell $(python-config) --cflags)

a.out: main.c transcendentals.so
    gcc-5 $(cflags) $(ldflags) transcendentals.c main.c

transcendentals.so: setup.py transcendentals.pyx
    python setup.py build_ext --inplace
    cython transcendentals.pyx


clean:
    rm -r a.out a.out.dSYM build transcendentals.[ch] transcendentals.so transcendentals_api.h

However, I came to error Segmentation fault: 11. Any idea can help with this? Thanks!

Answer 1

In that Makefile there is

transcendentals.so: setup.py transcendentals.pyx
    python setup.py build_ext --inplace

Unless python refers to /Users/spacegoing/anaconda/bin/python3 it should be replaced since the module may be compiled for wrong python version, and cannot thus be loaded.

In main.c there is call import_transcendentals() that does not check the return value i.e. if the import fails or succeeds. In case of failure, get_pi() and get_e() point to invalid memory locations and trying to call them causes a segmentation fault.

Also, the module has to be located somewhere where it can be found. It seems that when embedding, the current directory is not searched for python modules. PYTHONPATH environment variable could be changed to include the directory where transcendentals.so is located.

The following is an altenative way of embedding the code to the C program and sidesteps the import issues since the module code is linked to the executable.

Essentially, a call to PyInit_transcendentals() is missing.

File transcendentals.h will be generated when the cython functions are defined public i.e.

cdef public api double get_pi():
...
cdef public api double get_e():

Your main.c should have the include directives

#include <Python.h>
#include "transcendentals.h"

and then in main

Py_Initialize();
PyInit_transcendentals();

There should be no #include "transcendentals_api.h" and no import_transcendentals()

The first reason is that according to the documentation

However, note that you should include either modulename.h or modulename_api.h in a given C file, not both, otherwise you may get conflicting dual definitions.

The second reason is, that since transcendentals.c is linked to the program in

gcc $(cflags) $(ldflags) transcendentals.c main.c

there is no reason to import transcendentals module. The module has to be initialized though, PyInit_transcendentals() does that for Python 3