How to use pybind11 in multithreaded application

Question

I want to run Python in a worker thread. However I get strange segfaults and deadlocks of the threads in the worker pool. How do I correctly use pybind11/Python C API to allow the threads to run the jobs?

I know that it does not make much sense to MT python because of the GIL, but thats an intermediate solution to fit the current architecture until there is a better approach.

Answer 1

this works. Wrap the long running c++ code with gil_scoped_release and gil_scoped_acquire

pybind11::gil_scoped_release release;

while (true)
{
    // do something and break
}

pybind11::gil_scoped_acquire acquire;