Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Cython+distutils build on Ubuntu Python 3 changes the module lib name during linking

I'm using Cython and distutils. During one of my tests, a binary extension module is built. After this, one of my other tests will import the binary module and test it. On Travis-CI, the linker command changes the name. See in the screenshot below: the compiler correctly builds test01.o, but the linker incorrectly creates test01.cpython-34m.so: I want test01.so.

enter image description here

The above screenshot is for Python 3.4; the problem doesn't happen for Python 2.7, leading me to suspect that the Python-specific distutils may be doing something differently.

The distutils setup() function, which is called from my main script, does nothing special:

setup(
    cmdclass = {'build_ext': build_ext},
    ext_modules = ext_modules,
    include_dirs=[numpy.get_include()]
)

Here ext_modules is a sequence of Extension() classes, imported from distutils.extension.

I have gone throught the distutils docs for 2.7 and 3.4 and nothing jumps out at me as different. What could be causing the name change, and what options could I specify, either in setup() or Extension() instantiation to prevent the name change?


Edit: I have set up a local VM running Ubuntu 13.10 and can confirm that, as described above, Python 2.7 produces test01.so while Python 3.4 produces test01.cpython-34m.so. Therefore this issue has nothing to do with travis-ci and I am removing that tag and editing the title.


Edit: Indeed it is a change to distutils, made in 3.2. I'm still figuring out what I can do to get my import statement to look right.


Edit: OMG the helloworld example in the python docs produces helloworld.cpython-34m.so on Python 3.4.1 (Anaconda) on Ubuntu 13.10. Python's sysconfig module starts up with sysconfig.get_config_var('SO') == 'cpython-34m.so'.

I feel a rant coming on.

like image 964
Caleb Hattingh Avatar asked Aug 11 '14 02:08

Caleb Hattingh


1 Answers

Mystery solved, thanks to @ncoghlan_dev. The renaming change was made in PEP 3149. It turns out that even though the binary extension module might be called something like helloworld.cpython-34m.so, when you are actually in a python interpreter, import helloworld still works. My travis tests were failing because I had a test that checked for the name of the compiled .so file. If I remove that test, the rest of the file should still work.

like image 94
Caleb Hattingh Avatar answered Sep 21 '22 00:09

Caleb Hattingh