TensorFlow: PlaceHolder error when using tf.merge_all_summaries()

Question

I am getting a placeholder error.

I do not know what it means, because I am mapping correctly on sess.run(..., {_y: y, _X: X})... I provide here a fully functional MWE reproducing the error:

import tensorflow as tf
import numpy as np

def init_weights(shape):
    return tf.Variable(tf.random_normal(shape, stddev=0.01))

class NeuralNet:
    def __init__(self, hidden):
        self.hidden = hidden

    def __del__(self):
        self.sess.close()

    def fit(self, X, y):
        _X = tf.placeholder('float', [None, None])
        _y = tf.placeholder('float', [None, 1])

        w0 = init_weights([X.shape[1], self.hidden])
        b0 = tf.Variable(tf.zeros([self.hidden]))
        w1 = init_weights([self.hidden, 1])
        b1 = tf.Variable(tf.zeros([1]))

        self.sess = tf.Session()
        self.sess.run(tf.initialize_all_variables())

        h = tf.nn.sigmoid(tf.matmul(_X, w0) + b0)
        self.yp = tf.nn.sigmoid(tf.matmul(h, w1) + b1)

        C = tf.reduce_mean(tf.square(self.yp - y))
        o = tf.train.GradientDescentOptimizer(0.5).minimize(C)

        correct = tf.equal(tf.argmax(_y, 1), tf.argmax(self.yp, 1))
        accuracy = tf.reduce_mean(tf.cast(correct, "float"))
        tf.scalar_summary("accuracy", accuracy)
        tf.scalar_summary("loss", C)

        merged = tf.merge_all_summaries()
        import shutil
        shutil.rmtree('logs')
        writer = tf.train.SummaryWriter('logs', self.sess.graph_def)

        for i in xrange(1000+1):
            if i % 100 == 0:
                res = self.sess.run([o, merged], feed_dict={_X: X, _y: y})
            else:
                self.sess.run(o, feed_dict={_X: X, _y: y})
        return self

    def predict(self, X):
        yp = self.sess.run(self.yp, feed_dict={_X: X})
        return (yp >= 0.5).astype(int)


X = np.array([ [0,0,1],[0,1,1],[1,0,1],[1,1,1]])
y = np.array([[0],[1],[1],[0]]])

m = NeuralNet(10)
m.fit(X, y)
yp = m.predict(X)[:, 0]
print accuracy_score(y, yp)

The error:

I tensorflow/core/common_runtime/local_device.cc:40] Local device intra op parallelism threads: 8
I tensorflow/core/common_runtime/direct_session.cc:58] Direct session inter op parallelism threads: 8
0.847222222222
W tensorflow/core/common_runtime/executor.cc:1076] 0x2340f40 Compute status: Invalid argument: You must feed a value for placeholder tensor 'Placeholder_1' with dtype float
     [[Node: Placeholder_1 = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
W tensorflow/core/common_runtime/executor.cc:1076] 0x2340f40 Compute status: Invalid argument: You must feed a value for placeholder tensor 'Placeholder' with dtype float
     [[Node: Placeholder = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
Traceback (most recent call last):
  File "neuralnet.py", line 64, in <module>
    m.fit(X[tr], y[tr, np.newaxis])
  File "neuralnet.py", line 44, in fit
    res = self.sess.run([o, merged], feed_dict={self._X: X, _y: y})
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 368, in run
    results = self._do_run(target_list, unique_fetch_targets, feed_dict_string)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 444, in _do_run
    e.code)
tensorflow.python.framework.errors.InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder_1' with dtype float
     [[Node: Placeholder_1 = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
Caused by op u'Placeholder_1', defined at:
  File "neuralnet.py", line 64, in <module>
    m.fit(X[tr], y[tr, np.newaxis])
  File "neuralnet.py", line 16, in fit
    _y = tf.placeholder('float', [None, 1])
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/array_ops.py", line 673, in placeholder
    name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 463, in _placeholder
    name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/op_def_library.py", line 664, in apply_op
    op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1834, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1043, in __init__
    self._traceback = _extract_stack()

If I remove the tf.merge_all_summaries() or remove merged from self.sess.run([o, merged], ...) then it runs okay.

This looks similar to this post: Error when computing summaries in TensorFlow However, I am not using iPython...

Answer 1

The tf.merge_all_summaries() function is convenient, but also somewhat dangerous: it merges all summaries in the default graph, which includes any summaries from previous—apparently unconnected—invocations of code that also added summary nodes to the default graph. If old summary nodes depend on an old placeholder, you will get errors like the one you have shown in your question (and like previous questions as well).

There are two independent workarounds:

1. Ensure that you explicitly collect the summaries that you wish to compute. This is as simple as using the explicit tf.merge_summary() op in your example:

   accuracy_summary = tf.scalar_summary("accuracy", accuracy)
   loss_summary = tf.scalar_summary("loss", C)
   
   merged = tf.merge_summary([accuracy_summary, loss_summary])
   

2. Ensure that each time you create a new set of summaries, you do so in a new graph. The recommended style is to use an explicit default graph:

   with tf.Graph().as_default():
     # Build model and create session in this scope.
     #
     # Only summary nodes created in this scope will be returned by a call to
     # `tf.merge_all_summaries()`
   

   Alternatively, if you are using the latest open-source version of TensorFlow (or the forthcoming 0.7.0 release), you can call tf.reset_default_graph() to reset the state of the graph and remove any old summary nodes.