OS X Python: can I explicitly set MACOSX_DEPLOYMENT_TARGET for extensions?

Question

I use the python.org framework build of Python under recent versions of OS X (i.e., 10.11 El Capitan). I need to build some extensions that rely on recent versions of compilers (e.g., C++-11 features). However the python.org python is built to work on older systems as well, for backward compatibility.

Hence, it has the environment variable MACOSX_DEPLOYMENT_TARGET=10.6. This means that extensions are built by default with a toolchain that, I think, mimics gcc-4.2, in particular in terms of what stdlib it searches.

In the past, I have fixed this by installing more recent compilers with homebrew and explicitly setting CC, CXX, etc before installation.

However, I have tried just setting MACOSX_DEPLOYMENT_TARGET=10.11, and that seems to work. Is this safe? Are there any downsides? (I don't need to distribute these builds, just use them locally?)

Answer 1

The python.org OS X 64-bit/32-bit framework builds for recent Python releases were built with MACOSX_DEPLOYMENT_TARGET set to 10.6 and built on Mac OS X 10.6 to ensure compatibility with a wide range of OS X releases. Currently that range is from 10.6 Snow Leopard through 10.11 El Capitan. When building C or C++ extension modules using Python's build-in Distutils or high-level tools that use Distutils (such as pip), by default the deployment target for the compilation and link environment is set to that of the interpreter build, so in this case 10.6, to try to produce extension modules that will work with the same range of OS X releases as the interpreter build itself. There is seldom a need to change this since Apple generally is very good about maintaining backwards compatibility for the system libraries and frameworks used by Python itself. However, as you discovered, you can override the deployment target to a newer release by setting the MACOSX_DEPLOYMENT_TARGET environment variable prior to building the extension module. (Distutils checks for and does not allow setting the deployment target to a release older than that used for the interpreter build.) Distutils also honors overriding various other build values by setting the corresponding "standard" environment variables, such as CC, CXX, CFLAGS, LDSHARED, etc.

One case where you might need to change the deployment target is if you are dealing with C++ code. As has been widely discussed elsewhere, in recent releases Apple has been in the process of migrating from the GCC-based libstdc++ to the Clang/LLVM libc++ standard library for C++ programs. Apple has been shipping both. The Python interpreter and its supplied standard library do not use C++ at all so this issue does not affect Python itself. But if you use extension modules that are written in C++ (or link to a third-party library written in C++), either your own or from a third-party package (downloaded from PyPI, for example), you may need to be careful that all of the C++ code is either built using the same C++ standard library or, if not, that the differing C++ modules do not share objects. I don't have personal experience with such C++ situations so I'm not sure what is the best way to avoid any problems that might arise. But a quick-and-dirty check might be to use Apple's otool command line utility on all extension modules and the shared libraries and frameworks they link with to find all references to either libstdc++ and libc++, so something like starting with inspecting the output from:

find -E . -regex '(.*\.so)|(.*\.dylib)' -exec  otool -L '{}' ';'