Can two threads use the same embedded python interpreter simultaneously?

Question

The title has it, but here are some elaborations. Suppose the main thread spawns another thread, where some code is loaded into python interpreter and then another thread is called which executes some more code through the same python interface (through PyImport or PyRun). Is such scenario feasable?

Answer 1

If I'm following what you are asking, then yes you can do this, but the Python interpreter itself is not fully thread safe. To get around this, you must make sure each thread obtains the interpreter's GIL before calling any Python code and then releases it afterwards. i.e. Each thread needs to do the following when executing Python code:

PyGILState_STATE gstate;
gstate = PyGILState_Ensure();

// Do any needed Python API operations, execute python code

// Release the GIL. No Python API allowed beyond this point.
PyGILState_Release(gstate);

Also you should do the following after starting the Python interpreter to ensure threads/GIL are properly initialized:

if (! PyEval_ThreadsInitialized()) {
    PyEval_InitThreads();
}

See Non Python Created Threads for more info on this.

As mentioned in the comments, it's worth noting that this is really just serializing access to the interpreter, but it's the best you can do assuming you are using the CPython implementation of Python.