Cython unable to find shared object file

Question

I am trying to link to my own C library from Cython, following the directions I've found on the web, including this answer:

Using Cython To Link Python To A Shared Library

I am running IPython through Spyder.

My setup.py looks like this:

from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
import numpy as np

setup(
  ext_modules = cythonize(
      [Extension("*",["*.pyx"],
                 libraries =["MyLib"],
                 extra_compile_args = ["-fopenmp","-O3"],
                 extra_link_args=["-L/path/to/lib"])
                 ]),
  include_dirs = [np.get_include()],
)

The file libMyLib.so is in /path/to/lib and it compiles fine.

I have a Python script in my IPython profile startup folder that does this

try:
  os.environ["LD_LIBRARY_PATH"] += ":/path/to/lib"
except KeyError:
  os.environ["LD_LIBRARY_PATH"] = "/path/to/lib"

I can confirm that this is running, because if I type os.environ["LD_LIBRARY_PATH"] into the IPython interpreter, it returns /path/to/lib

But when I try to load the Cython module (i.e. import mycythonmodule) I get:

ImportError: libMyLib.so: cannot open shared object file: No such file or directory

I've also tried putting libMyLib.so in other places to see if cython would find it:

• In the directory where Python is running
• On the Python path
• In the same folder as the cython module

But it still doesn't find the shared library. The only way I can get it to find the library is by dropping it in /usr/lib, but I don't want it there, I want to be able to set the library path.

Am I missing something?

Answer 1

I'm self-answering, in case anyone else runs into the same problem. Looks like the answers are here:

Set LD_LIBRARY_PATH before importing in python

Changing LD_LIBRARY_PATH at runtime for ctypes

According to these answers (and my experience), the linker reads LD_LIBRARY_PATH when python is launched, so changing it from within python doesn't have any useful effect, at least not the effect I was hoping for. The only solution is to either wrap python in a shell script that sets LD_LIBRARY_PATH, or else drop the shared object somewhere on the linker search path.

Kind of a pain, but it is what it is.

Answer 2

I have fixed it by change setup.py.

1. I have a C++ dynamic shared library called "libtmsmdreader.so". and a header file named "TmsMdReader.hpp"
2. I wrapper C++ shared library to cython library called "tmsmdreader-pythonxxxxxx.so"

from setuptools import setup 
from distutils.extension import Extension
from Cython.Build import cythonize

setup(
    name="tmsmdreader",
    ext_modules=cythonize([
        Extension(
            name="tmsmdreader",
            language="c++",
            sources=["TmsMdReaderApi.pyx"],
            libraries=["tmsmdreader"],
            library_dirs=["."],
            include_dirs=["."],
            extra_compile_args=["-std=c++14"],
            compiler_directives={'language_level': 3},
            runtime_library_dirs=["."])
            ]))

library_dirs=["."] and runtime_library_dirs=["."] can fixed LD_LIBRARY_PATH if libtmsmdreader.so in python scripy directory