Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python setuptools not including C++ standard library headers

I'm trying to compile a Python wrapper to a small C++ library I've written. I've written the following setup.py script to try to use setuptools to compile the wrapper:

from setuptools import setup, Extension
import numpy as np
import os

atmcmodule = Extension(
    'atmc',
    include_dirs=[np.get_include(), '/usr/local/include'],
    libraries=['mcopt', 'c++'],  # my C++ library is at ./build/libmcopt.a
    library_dirs=[os.path.abspath('./build')],
    sources=['atmcmodule.cpp'],
    language='c++',
    extra_compile_args=['-std=c++11', '-v'],
    )

setup(name='tracking',
      version='0.1',
      description='Particle tracking and MC optimizer module',
      ext_modules=[atmcmodule],
      )

However, when I run python setup.py build on OS X El Capitan, clang complains about not finding some C++ standard library headers:

In file included from atmcmodule.cpp:7:
In file included from ./mcopt.h:11:
In file included from ./arma_include.h:4:
/usr/local/include/armadillo:54:12: fatal error: 'initializer_list' file not found
  #include <initializer_list>
           ^
1 error generated.
error: command 'gcc' failed with exit status 1

Passing the -v flag to the compiler shows that it is searching the following include paths:

#include <...> search starts here:
 /Users/[username]/miniconda3/include
 /Users/[username]/miniconda3/lib/python3.4/site-packages/numpy/core/include
 /usr/local/include
 /Users/[username]/miniconda3/include/python3.4m
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/c++/4.2.1
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/c++/4.2.1/backward
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/7.0.0/include
 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/System/Library/Frameworks (framework directory)
End of search list.

This apparently doesn't include the path to the C++ standard library headers. If I compile a small test C++ source with the -v option, I can see that clang++ normally also searches the path /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1, and if I include this path in the include_dirs option for Extension in my setup.py script, then the extension module compiles correctly and works. However, hard-coding this path into the script doesn't seem like a good solution since this module also needs to work on Linux.

So, my question is how do I properly make setuptools include the required headers?

Update (11/22/2015)

As setuptools tries to compile the extension, it prints the first command it's running:

gcc -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/[username]/miniconda3/include -arch x86_64 -I/Users/[username]/miniconda3/lib/python3.4/site-packages/numpy/core/include -I/Users/[username]/Documents/Code/ar40-aug15/monte_carlo/mcopt -I/usr/local/include -I/Users/[username]/miniconda3/include/python3.4m -c /Users/[username]/Documents/Code/ar40-aug15/monte_carlo/atmc/atmcmodule.cpp -o build/temp.macosx-10.5-x86_64-3.4/Users/[username]/Documents/Code/ar40-aug15/monte_carlo/atmc/atmcmodule.o -std=c++11 -fopenmp -v

If I paste this command into a terminal and run it myself, the extension compiles successfully. So I suspect either setuptools is modifying some environment variables I'm not aware of, or it's lying a little about the commands it's actually running.

like image 871
jb326 Avatar asked Nov 16 '15 15:11

jb326


1 Answers

Setuptools tries to compile C/C++ extension modules with the same flags used to compile the Python interpreter. After checking the flags used to compile my Python install (from Anaconda), I found it was compiling for a minimum Mac OS X version of 10.5. This seems to make it use the GCC libstdc++ instead of clang's libc++ (which supports C++11).

This can be fixed by either setting the environment variable MACOSX_DEPLOYMENT_TARGET to 10.9 (or later), or adding '-mmacosx-version-min=10.9' to extra_compile_args.

like image 183
jb326 Avatar answered Nov 01 '22 12:11

jb326