How to embed Python3 with the standard library

Question

I am attempting to embed Python in an (ultimately multiplatform) C++ app.

It's important that my app contains its own Python implementation (in the same way that blender does), so that it is entirely self-contained. (Else it becomes a configuration minefield).

I have two options:

1. Attempt to embed Python3 without the standard library (which I have asked here)

2. Attempt to embed Python3 with the standard library.

What is required for (2)?

With this information I would be able to balance the merits of each approach against the effort required to set it up.

My embedded Python will be for my own use (rather than any userland scripting) -- mainly control flow / game logic. I will need very little from the standard library -- maybe I can whittle that down to 0 by tunnelling back into C++ whenever necessary -- for example if I need a random number, I can create a C++ routine and access that from Python. I have all of that covered.

However, it is starting to look as though even a minimal installation will have to contain some stdlib component(s), which prompts the question: "If I must include some, maybe it is better to include all!"

Answer 1

Since this doesn't really have an answer, I will offer this for posterity. I also do not have access to a Mac, so it may be a little different for you than on Linux. Also, the required dependencies must be installed for this to work, but that is usually easy enough to figure out.

Create a working directory

mkdir ~/embeddedpython
cd ~/embeddedpython

Download the Python source

wget https://www.python.org/ftp/python/3.6.1/Python-3.6.1.tgz

Create an installation directory for Python

mkdir ./installation

Extract the downloaded source files

tar xvf Python-3.6.1.tgz

Enter the newly created source directory

cd Python-3.6.1

Configure Python to install in our installation directory

./configure --prefix="/home/<username>/embeddedpython/installation"

Make and install Python

make && make install

Go back to your working directory

cd ..

Create a new PYTHONHOME directory where the library will reside

mkdir home && mkdir home/lib

Copy the Python library to our new home directory

cp -r ./installation/lib/python3.6 ./home/lib/

Create a new c++ source file (embeddedpython.cpp) with the following code taken from the python documentation, with the exception of the setenv function call.

#include <Python.h>
#include <cstdlib>

int main(int argc, char *argv[])
{
    setenv("PYTHONHOME", "./home", 1);

    wchar_t *program = Py_DecodeLocale(argv[0], NULL);
    if (program == NULL) {
        fprintf(stderr, "Fatal error: cannot decode argv[0]\n");
        exit(1);
    }
    Py_SetProgramName(program);  /* optional but recommended */
    Py_Initialize();
    PyRun_SimpleString("from time import time,ctime\n"
                       "print('Today is', ctime(time()))\n");
    if (Py_FinalizeEx() < 0) {
        exit(120);
    }
    PyMem_RawFree(program);
    return 0;
}

Compile and run

g++ embeddedpython.cpp -I ./installation/include/python3.6m/ ./installation/lib/libpython3.6m.a  -lpthread -ldl -lutil
./a.out

> Today is Fri Apr 14 16:06:54 2017

From here on it is standard embedded python as usual. With this method, the "home" directory must be included in your deployment, and the environment variable PYTHONHOME must be set to point to it before any python related code is executed.

Answer 2

You are looking for Boost.Python!

It's a C++ library which enables seamless interoperability between C++ and the Python programming language and in my opinion this should suffice your need, unless you are trying to achieve something else.

It also has a mechanism for embedding the python interpreter into C++ code and one can refer to this link (URL isn't release specific) to delve into the possibilities.

P.S. I believe less in reinventing the wheel and more into the re-usability.

Answer 3

I suppose that you have already double check how to Embedding Python in Another Application (here you will see something which cover embedding python2 but will be true for python3 also in my opinion)

There is different types of embedding:

• Very High Level Embedding
• Beyond Very High Level Embedding
• Pure Embedding
• Embedding Python in C++

As your question is relative to "Embedding Python in C++" you may read this:

It is also possible to embed Python in a C++ program; precisely how this is done will depend on the details of the C++ system used; in general you will need to write the main program in C++, and use the C++ compiler to compile and link your program. There is no need to recompile Python itself using C++.

As one hand you said "(ultimately multiplatform) C++ app", and in the other hand you have "precisely how this is done will depend on the details of the C++ system used", so may you explain more details of the C++ system used ?

You may also find some tips here relative to the use of pybind11 module or another old page which treat about how to Embed Python and Import Modules in C/C++ (python2.6 but I hope you may found inspiration with)

To conclude:

You'll obviously need development packages for Python in order to have the Python include directory