How to call a python function from a foreign language thread (C++)

Question

I am developing a program that use DirectShow to grab audio data from media files. DirectShow use thread to pass audio data to the callback function in my program, and I let that callback function call another function in Python.

I use Boost.Python to wrapper my library, the callback function :

class PythonCallback {
private:
    object m_Function;
public:
    PythonCallback(object obj)
        : m_Function(obj)
    {}

    void operator() (double time, const AudioData &data) {
        // Call the callback function in python
        m_Function(time, data);
    }

};

Here comes the problem, a thread of DirectShow calls my PythonCallback, namely, call the function in Python. Once it calls, my program just crash. I found this should be threading problem. Then I found this document:

http://docs.python.org/c-api/init.html

It seems that my program can't call to Python's function from thread directly, because there is Global Interpreter Lock. The python's GIL is so complex, I have no idea how it works. I'm sorry, what I can do is to ask. My question is. What should I do before and after I call a Python function from threads?

It may looks like this.

void operator() (double time, const AudioData &data) {
    // acquire lock
    m_Function(time, data);
    // release lock
}

Thanks. Victor Lin.

Answer 1

Take a look at PyGILState_Ensure()/PyGILState_Release(), from PEP 311 http://www.python.org/dev/peps/pep-0311/

Here is an example taken from the PEP itself:

void SomeCFunction(void)
{
    /* ensure we hold the lock */
    PyGILState_STATE state = PyGILState_Ensure();
    /* Use the Python API */
    ...
    /* Restore the state of Python */
    PyGILState_Release(state);
}

Answer 2

Have the c++ callback place the data in a queue. Have the python code poll the queue to extract the data.