Tensorflow Enqueue operation was cancelled

Question

I had built a convolutional neural network in tensorflow. It is trained and now I am unpacking it and performing evaluations.

import main
import Process
import Input

eval_dir = "/Users/Zanhuang/Desktop/NNP/model.ckpt-250"
checkpoint_dir = "/Users/Zanhuang/Desktop/NNP/checkpoint"

def evaluate():
  with tf.Graph().as_default() as g:
    images, labels = Process.eval_inputs()
    forward_propgation_results = Process.forward_propagation(images)
    init_op = tf.initialize_all_variables()
    saver = tf.train.Saver()
    top_k_op = tf.nn.in_top_k(forward_propgation_results, labels, 1)

  with tf.Session(graph=g) as sess:
    tf.train.start_queue_runners(sess=sess)
    sess.run(init_op)
    saver.restore(sess, eval_dir)
    print(sess.run(top_k_op))


def main(argv=None):
    evaluate()

if __name__ == '__main__':
  tf.app.run()

Unfortunately a strange error has popped up and I have no clue why.

W tensorflow/core/kernels/queue_base.cc:2
W tensorflow/core/kernels/queue_base.cc:294] _0_input_producer: Skipping cancelled enqueue attempt with queue not closed
W tensorflow/core/kernels/queue_base.cc:294] _1_batch/fifo_queue: Skipping cancelled enqueue attempt with queue not closed
E tensorflow/core/client/tensor_c_api.cc:485] Enqueue operation was cancelled
     [[Node: batch/fifo_queue_enqueue = QueueEnqueue[Tcomponents=[DT_FLOAT, DT_INT32], _class=["loc:@batch/fifo_queue"], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/fifo_queue, Cast_1, Cast)]]
E tensorflow/core/client/tensor_c_api.cc:485] Enqueue operation was cancelled
     [[Node: batch/fifo_queue_enqueue = QueueEnqueue[Tcomponents=[DT_FLOAT, DT_INT32], _class=["loc:@batch/fifo_queue"], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/fifo_queue, Cast_1, Cast)]]
E tensorflow/core/client/tensor_c_api.cc:485] Enqueue operation was cancelled
     ....
     [[Node: batch/fifo_queue_enqueue = QueueEnqueue[Tcomponents=[DT_FLOAT, DT_INT32], _class=["loc:@batch/fifo_queue"], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/fifo_queue, Cast_1, Cast)]]
E tensorflow/core/client/tensor_c_api.cc:485] Enqueue operation was cancelled
     [[Node: batch/fifo_queue_enqueue = QueueEnqueue[Tcomponents=[DT_FLOAT, DT_INT32], _class=["loc:@batch/fifo_queue"], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch/fifo_queue, Cast_1, Cast)]]
W tensorflow/core/kernels/queue_base.cc:294] _1_batch/fifo_queue: Skipping cancelled enqueue attempt with queue not closed
...
W tensorflow/core/kernels/queue_base.cc:294] _1_batch/fifo_queue: Skipping cancelled enqueue attempt with queue not closed
E tensorflow/core/client/tensor_c_api.cc:485] Enqueue operation was cancelled

This is only a part of it.

Answer 1

Update from chat -- the program runs successfully, and the messages that are printed are due to Python killing threads while they are running as the process exits.

The messages are harmless but it's possible to avoid them by stopping threads manually using pattern below.

coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(sess=sess, coord=coord)
<do stuff>
coord.request_stop()
coord.join(threads)