Menu
I'm using Python and Keras (currently using Theano backend, but I have no qualms with switching). I have a neural network that I load and process multiple sources of information with in parallel. Currently, I've been running each one in a separate process and it loads its own copy of the network from the file. This seems like a waste of RAM, so I was thinking it would be more efficient to have a single multi-threaded process with one instance of the network that is used by all threads. However, I'm wondering if Keras is thread safe with either backend. If I run .predict(x) on two different inputs at the same time in different threads, will I run into race conditions or other issues?
Thanks
775
Tensorflow.NET is thread-safe, our multithreading model is thread-wide Session and Graph; meaning tf. get_default_graph/session() are unique to the thread they are executed in.
Python is not by its self thread safe. But there are moves to change this: NoGil, etc. Removing the GIL does not make functions thread-safe.
An object is thread-safe for reading from multiple threads. For example, given an object A, it is safe to read A from thread 1 and from thread 2 simultaneously. If an object is being written to by one thread, then all reads and writes to that object on the same or other threads must be protected.
the standard C printf() and scanf() functions use stdio so they are thread-safe.
Yes, Keras is thread safe, if you pay a little attention to it.
In fact, in reinforcement learning there is an algorithm called Asynchronous Advantage Actor Critics (A3C) where each agent relies on the same neural network to tell them what they should do in a given state. In other words, each thread calls model.predict concurrently as in your problem. An example implementation with Keras of it is here.
You should, however, pay extra attention to this line if you looked into the code: model._make_predict_function() # have to initialize before threading
This is never mentioned in the Keras docs, but its necessary to make it work concurrently. In short, _make_predict_function is a function that compiles the predict function. In multi thread setting, you have to manually call this function to compile predict in advance, otherwise the predict function will not be compiled until you run it the first time, which will be problematic when many threading calling it at once. You can see a detailed explanation here.
I have not met any other issues with multi threading in Keras till now.
117
to quote the kind fcholet:
_make_predict_function is a private API. We should not recommend calling it.
Here, the user should simply call predict first.
Note that Keras models can't be guaranteed to be thread-safe. Consider having independent copies of the model in each thread for CPU inference.
29
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
