Stop embedded python

Question

I'm embedding python in a C++ plug-in. The plug-in calls a python algorithm dozens of times during each session, each time sending the algorithm different data. So far so good

But now I have a problem: The algorithm takes sometimes minutes to solve and to return a solution, and during that time often the conditions change making that solution irrelevant. So, what I want is to stop the running of the algorithm at any moment, and run it immediately after with other set of data.

Here's the C++ code for embedding python that I have so far:

void py_embed (void*data){


counter_thread=false;

PyObject *pName, *pModule, *pDict, *pFunc;

//To inform the interpreter about paths to Python run-time libraries
Py_SetProgramName(arg->argv[0]);

if(!gil_init){
    gil_init=1;
    PyEval_InitThreads();
    PyEval_SaveThread();
}
PyGILState_STATE gstate = PyGILState_Ensure();

// Build the name object
pName = PyString_FromString(arg->argv[1]);
if( !pName ){
    textfile3<<"Can't build the object "<<endl;
}

// Load the module object
pModule = PyImport_Import(pName);
if( !pModule ){
    textfile3<<"Can't import the module "<<endl;
}

// pDict is a borrowed reference 
pDict = PyModule_GetDict(pModule);
if( !pDict ){
    textfile3<<"Can't get the dict"<<endl;
}

// pFunc is also a borrowed reference 
pFunc = PyDict_GetItemString(pDict, arg->argv[2]);
if( !pFunc || !PyCallable_Check(pFunc) ){
    textfile3<<"Can't get the function"<<endl;
}

/*Call the algorithm and treat the data that is returned from it
 ...
 ...
 */

// Clean up
Py_XDECREF(pArgs2);
Py_XDECREF(pValue2);
Py_DECREF(pModule);
Py_DECREF(pName);

PyGILState_Release(gstate);

counter_thread=true;
_endthread(); 

};

Edit: The python's algorithm is not my work and I shouldn't change it

Answer 1

This is based off of a cursory knowledge of python, and reading the python docs quickly.

PyThreadState_SetAsyncExc lets you inject an exception into a running python thread.

Run your python interpreter in some thread. In another thread, PyGILState_STATE then PyThreadState_SetAsyncExc into the main thread. (This may require some precursor work to teach the python interpreter about the 2nd thread).

Unless the python code you are running is full of "catch alls", this should cause it to terminate execution.

You can also look into the code to create python sub-interpreters, which would let you start up a new script while the old one shuts down.

Py_AddPendingCall is also tempting to use, but there are enough warnings around it maybe not.