How to embed python in an Objective-C OS X application for plugins?

Question

I'm trying to use python in a new OS X application for plugin scripting. I'm looking to offload some of the program logic to python for easy, on-the-fly modification. I had started by looking at the Big Nerd Ranch tutorial. That seemed to work, but it suggested an older method of linking Python. It appears that since Xcode 5, we are supposed to install python using this Mac Developer Library Tech Note. This process, however, seems to create a linked python instance, not an embedded one. I've tried following the guidelines in the answer to this question but it seems to break down.

So my question is this: what are the current best practices for building python for use as a plugin runtime in an Objective C Mac OS X app? How does one go about making sure that it is bundled with the application, and that it is possible to install any additional libraries that one might want before the final Objective C app is built?

Answer 1

I've come up with a method that seems to work fairly well.

First, download source of the python version you want to use from the official website. Extract the archive somewhere. I'm using Python 3.4.2. Adjust the commands on your system for the specific version you're using.

Create a build directory that you will use for this development python version. The entire directory should have no spaces in it to make sure that bash interprets the she-bang (#!) lines correctly. I used /Users/myaccount/development/python/devbuild/python3.4.2.

Go into the extracted Python directory and run the following commands:

./configure --prefix="/Users/myaccount/development/python/devbuild/python3.4.2"
make
make install

This will install python in that development build directory. Setup the Python path to use the correct directory:

export PYTHONPATH="/Users/myaccount/development/python/devbuild/python3.4.2/lib/python3.4/site-packages/"

Go into the python bin directory (/Users/myaccount/development/python/devbuild/python3.4.2/bin) and use the pip3 there to install any modules that you need. The $PYTHONPATH setting will ensure that the modules get installed into the correct site-packages directory.

Find a handy home for the PyObjC repository and clone it there. Then checkout the latest version tag and install it, making sure that your $PYTHONPATH is still correct:

hg clone https://bitbucket.org/ronaldoussoren/pyobjc
cd pyobjc
hg tags
hg checkout [the id of the latest version from the tag list]
/Users/myaccount/development/python/devbuild/python3.4.2/bin/python3 ./install.py

Whenever you need to update the python modules, just make sure to use the correct python bin and $PYTHONPATH.

Now to add python to an Xcode project.

Drag the /Users/myaccount/development/python/devbuild/python3.4.2 directory into the Xcode project, setting it to not copy items as necessary, and to create a folder reference.

Add /Users/myaccount/development/python/devbuild/python3.4.2/include/python3.4m to the Header Search Paths setting in the Xcode project's Build Settings. Not sure if there's a way to do this as a generalized step to just search the folder referenced directory we had just added.

Drag the `/Users/myaccount/development/python/devbuild/python3.4.2/lib/libpython3.4m.a library into the Xcode project, setting it to be added as a reference without copying as well.

The code from the Big Nerd Ranch scripting tutorial repository can now be used with a few modifications.

The Plugin Manager code will need an NSString extension to work with the wchar_t strings that the Python API seems to like so much:

@interface NSString (WcharEncodedString)

- (wchar_t*) getWideString;

@end

@implementation NSString (WcharEncodedString)

- (wchar_t*) getWideString {
    const char* tmp = [self cStringUsingEncoding:NSUTF8StringEncoding];
    unsigned long buflen = strlen(tmp) + 1;
    wchar_t* buffer = malloc(buflen * sizeof(wchar_t));
    mbstowcs(buffer, tmp, buflen);
    return buffer;
}

@end

The Python header should be included as follows:

#include "Python.h"

The following code needs to be run before Py_Initialize() is called in order to set up the correct python executable, PYTHONPATH, and PYTHONHOME as suggested by Zorg on that other question.

NSString* executablePath = [[NSBundle mainBundle] pathForResource:@"python3.4" ofType:nil inDirectory:@"python3.4.2/bin"];
Py_SetProgramName([executablePath getWideString]);

NSString* pythonDirectory = [[NSBundle mainBundle] pathForResource:@"python3.4" ofType:nil inDirectory:@"python3.4.2/lib"];
Py_SetPath([pythonDirectory getWideString]);
Py_SetPythonHome([pythonDirectory getWideString]);

Finally, the python path needs to be expanded in the PluginExecutor.py file to include the various subdirectories of the high level lib path. Add the following code to the top of the Plugin Executor file:

import sys
from os import walk
path = sys.path.copy()
for p in path:
    for root,dirs,files in walk(p):
        if p is not root:
            sys.path.append(root)

I'll post updates if things start to break down, but this seems a working solution for now.