Python interpreter as a c++ class

Question

I am working on embedding python in to c++. In some peculiar case I require two separate instances of the interpreter in same thread.

Can I wrap Python interpreter in to a c++ class and get services from two or more class instances?

Answer 1

mosaik's answer did not work in my situation where my module is a plugin to a host application that already initializes python. I was able to get it to work with the following code.

// initialize interpreter
::PyEval_InitThreads();
::PyThreadState *mainThread = ::PyThreadState_Get();
myState = ::Py_NewInterpreter();
... // call python code
::PyThreadState_Swap(mainThread);

... // any other code

mainThread = ::PyThreadState_Swap(myState)
... // call python code
::PyThreadState_Swap(mainThread)

... // any other code

// finished with interpreter
mainThread = ::PyThreadState_Swap(myState)
::Py_EndInterpreter(myState);
::PyThreadState_Swap(mainThread)

When I called PyEval_AcquireLock() the program blocked and the function did not return. Further, calling PyEval_ReleaseThread(myState) seemed to invalidate the interpreter also.

Answer 2

I have used Py_NewInterpreter for different interpreters in different threads, but this should also work for several interpreters within one thread:

In the main thread:

Py_Initialize();
PyEval_InitThreads();
mainThreadState = PyEval_SaveThread();

For each interpreter instance (in any thread):

// initialize interpreter
PyEval_AcquireLock();                // get the GIL
myThreadState = Py_NewInterpreter();
... // call python code
PyEval_ReleaseThread(myThreadState); // swap out thread state + release the GIL

... // any other code

// continue with interpreter
PyEval_AcquireThread(myThreadState); // get GIL + swap in thread state
... // call python code
PyEval_ReleaseThread(myThreadState);

... // any other code

// finish with interpreter
PyEval_AcquireThread(myThreadState);
... // call python code
Py_EndInterpreter(myThreadState);
PyEval_ReleaseLock();                // release the GIL

Note that you need a variable myThreadState for each interpreter instance!

Finally the finish in the main thread:

PyEval_RestoreThread(mainThreadState);
Py_Finalize();

There are some restrictions with using several interpreter instances (they seem not to be totally independent), but in most cases this does not seem to cause problems.