How do I pass a scalar via a TensorFlow feed dictionary

Question

My TensorFlow model uses tf.random_uniform to initialize a variable. I would like to specify the range when I begin training, so I created a placeholder for the initialization value.

init = tf.placeholder(tf.float32, name="init")
v = tf.Variable(tf.random_uniform((100, 300), -init, init), dtype=tf.float32)
initialize = tf.initialize_all_variables()

I initialize variables at the start of training like so.

session.run(initialize, feed_dict={init: 0.5})

This gives me the following error:

ValueError: initial_value must have a shape specified: Tensor("Embedding/random_uniform:0", dtype=float32)

I cannot figure out the correct shape parameter to pass to tf.placeholder. I would think for a scalar I should do init = tf.placeholder(tf.float32, shape=0, name="init") but this gives the following error:

ValueError: Incompatible shapes for broadcasting: (100, 300) and (0,)

If I replace init with the literal value 0.5 in the call to tf.random_uniform it works.

How do I pass this scalar initial value via the feed dictionary?

Answer 1

TL;DR: Define init with a scalar shape as follows:

init = tf.placeholder(tf.float32, shape=(), name="init")

This looks like an unfortunate implementation detail of tf.random_uniform(): it currently uses tf.add() and tf.multiply() to rescale the random value from [-1, +1] to [minval, maxval], but if the shape of minval or maxval is unknown, tf.add() and tf.multiply() can't infer the proper shapes, because there might be broadcasting involved.

By defining init with a known shape (where a scalar is () or [], not 0), TensorFlow can draw the proper inferences about the shape of the result of tf.random_uniform(), and your program should work as intended.

Answer 2

You don't need a placeholder to pass the scalar since any tensor, sparsetensor or nexted tuple of tensor or sparsetensor can be ok. The doc reads that:

The optional `feed_dict` argument allows the caller to override
the value of tensors in the graph. Each key in `feed_dict` can be
one of the following types:
* If the key is a `tf.Tensor`, the
  value may be a Python scalar, string, list, or numpy ndarray
  that can be converted to the same `dtype` as that
  tensor. Additionally, if the key is a
  `tf.placeholder`, the shape of
  the value will be checked for compatibility with the placeholder.
* If the key is a
  `tf.SparseTensor`,
  the value should be a
  `tf.SparseTensorValue`.
* If the key is a nested tuple of `Tensor`s or `SparseTensor`s, the value
  should be a nested tuple with the same structure that maps to their
  corresponding values as above.

And in your scenario, any tensor, like constant or variable or placeholder, can be appropriate.

init = tf.constant(0)
init_1 = tf.Variable(0)
v = tf.Variable(tf.random_uniform((100, 300), -init, init), dtype=tf.float32)
initialize = tf.global_variables_initializer()
sess.run(intialize, feed_dict={init: 0.5})
sess.run(intialize, feed_dict={init_1: 0.5})

You can pass either float or int to it because only placeholder checks data type as stated above.