What did Apple do to the Python framework?

Question

[tl;dr? see bottom]

Python on OS X has always been somewhat of an abomination in that it's split up and spread out all across the system. Some in /usr, some in /Library/Python, some /System/Library/Frameworks/Python.framework.

Now, as I understand it, the framework is supposed to contain the Python distribution, i.e. the bits and pieces that aren't going to change. An example would be headers, the standard library, the binary images, etc.

So as a developer of a sort-of-popular Python C extension, I think myself to be pretty good at the OS X ecosystem and how to compile Python extensions on it. It wasn't a month ago that Apple decided to skip on QA, breaking C extension building across the board.

It's broken in yet a new way though, as Apple seem to have decided to remove the better part of the Python distribution. Take a look at the include/ directory:

$ ls -l /System/Library/Frameworks/Python.framework/Versions/Current/include/python2.6 
total 16
-rw-r--r--  1 root  wheel    30K Jun 25  2010 pyconfig.h

Missing something? The Python.h header perhaps? What's more, I had woes with zc.buildout because it couldn't find site.py... Have a look-see:

$ python -c 'print __import__("site").__file__'
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site.pyc

$ ls -l /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site.py*
-rw-r--r--  1 root  wheel    20K May 17 15:40 /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site.pyc

Missing something? The site.py original perhaps?

All in all, it seems Apple are stripping out vital developer resources. I confirmed both of these findings on other MacBooks with OS X 10.6.7.

tl;dr Apple have removed lots of vital headers and source-code from the Python framework. Has this happened anybody else? If so, when did this happen? Why did it happen? And most importantly, how do I get them back?

Answer 1

Python.h and other headers are included with Xcode. On my system, it's located in both /Developer/SDKs/MacOSX10.6.sdk/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6/Python.h and /System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6/Python.h. The latter appears to be installed by the Xcode installer.

Answer 2

Actually, it doesn't look like you understand the OS X ecosystem at all. /System/Library is for Apple-shipped components of the OS X "distribution"; third parties must not put files in there, but should use /Library instead. That's why the built-in Pythons (multiple versions) are in /System/Library/Frameworks/Python.framework, but place the site-packages directories in /Library/Python, so that third-party modules can be installed there. As for /usr/bin, it contains symlinks to the python executable by version, and programs that autoselect the version based on some parameters (see man python). It's actually quite logical and tidy.

Now to answer your questions. C headers and other developer resources are not included in the default install of OS X. This is not specific to Python, and was done to save space on a default install. To get the developer resources, you need to install the developer tools. What were you going to do with a C header without a C compiler, anyway?

Answer 3

I have encountered similar problems in the past. Space_C0wb0y's suggestion works for me, and using python_select I can switch between "default" versions of python. This also de-couples me from XCode. I've installed versions of 2.6 and 2.7 using MacPorts, which by default places the distribution in locations such as /opt/local/Library/Frameworks/Python.framework/Versions/2.7. It is the only way I have managed to have consistent installations, including 3rd party modules.