Is it possible to set a timeout when dequeuing an item from a TensorFlow queue?

Question

I am building a neural network ensemble where each net reads the inputs from an input queue and writes its prediction to an output queue, and a separate client pushes a new input to all input queues then pulls the individual predictions from each output queue and aggregates them to produce the ensemble's prediction.

I would like the system to be resilient to a slow (or crashed) neural network client, so I need to set a timeout when pulling from each output queue. Ideally, the aggregator graph would behave nicely and just ignore that prediction.

The only solution I have found to have a dequeue timeout is to set the operation_timeout_in_ms configuration option when creating the session, but this applies to all operations in the graph (for this session). Not very granular.

Any other option?

Answer 1

You can set a timeout on an individual tf.Session.run() call, which is most useful when you have a potentially blocking operation, such as a dequeue(). To do this, pass an optional tf.RunOptions object to the run() call, and set the timeout_in_ms field to the desired timeout in milliseconds:

op = ...  # Assume this depends on dequeuing a tensor from a queue.
sess = tf.Session()

# Set a 10-second timeout.
run_options = tf.RunOptions(timeout_in_ms=10000)
try:
  sess.run(op, options=run_options)
except tf.errors.DeadlineExceededError:  # This will be raised if the timeout expires.
  # ...