How to get the dimensions of a tensor (in TensorFlow) at graph construction time?

Question

I am trying an Op that is not behaving as expected.

graph = tf.Graph()
with graph.as_default():
  train_dataset = tf.placeholder(tf.int32, shape=[128, 2])
  embeddings = tf.Variable(
    tf.random_uniform([50000, 64], -1.0, 1.0))
  embed = tf.nn.embedding_lookup(embeddings, train_dataset)
  embed = tf.reduce_sum(embed, reduction_indices=0)

So I need to know the dimensions of the Tensor embed. I know that it can be done at the run time but it's too much work for such a simple operation. What's the easier way to do it?

Answer 1

I see most people confused about tf.shape(tensor) and tensor.get_shape() Let's make it clear:

1. tf.shape

tf.shape is used for dynamic shape. If your tensor's shape is changable, use it. An example: a input is an image with changable width and height, we want resize it to half of its size, then we can write something like:
new_height = tf.shape(image)[0] / 2

2. tensor.get_shape

tensor.get_shape is used for fixed shapes, which means the tensor's shape can be deduced in the graph.

Conclusion: tf.shape can be used almost anywhere, but t.get_shape only for shapes can be deduced from graph.

Answer 2

Tensor.get_shape() from this post.

From documentation:

c = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
print(c.get_shape())
==> TensorShape([Dimension(2), Dimension(3)])

Answer 3

A function to access the values:

def shape(tensor):
    s = tensor.get_shape()
    return tuple([s[i].value for i in range(0, len(s))])

Example:

batch_size, num_feats = shape(logits)

Answer 4

Just print out the embed after construction graph (ops) without running:

import tensorflow as tf

...

train_dataset = tf.placeholder(tf.int32, shape=[128, 2])
embeddings = tf.Variable(
    tf.random_uniform([50000, 64], -1.0, 1.0))
embed = tf.nn.embedding_lookup(embeddings, train_dataset)
print (embed)

This will show the shape of the embed tensor:

Tensor("embedding_lookup:0", shape=(128, 2, 64), dtype=float32)

Usually, it's good to check shapes of all tensors before training your models.