Tensorflow what does "Incomplete shape" mean?

Question

I am reviewing tensorflow log and found the following line:

4 ops no flops stats due to incomplete shapes. Consider passing run_meta to use run_time shapes.

This message appears to be coming from the following code :

for op in graph.get_operations():
    try:
      stats = ops.get_stats_for_node_def(
          graph, op.node_def, REGISTERED_FLOP_STATS)
    except ValueError:
      # Catch Exception When shape is incomplete. Skip it.
      op_missing_shape += 1
      stats = None
  ......
  if op_missing_shape > 0 and not run_meta:
    sys.stderr.write('%d ops no flops stats due to incomplete shapes.\n' %
                     op_missing_shape)

In a similar situation with GRU, this line in log does not appear. So I assume that error is not caused by batch size. Could you please explain what is it? Also how do I add "run_meta" attribute? Thanks.

Answer 1

' ... no flops stats due to incomplete shapes' means that you have unknown [shape] information for some variables, e.g. when you process variable batch size ([None] shape) or a tf.while_loop for arbitrary amount of time, etc.

Accordingly to official tfprof docs (source):

• It must have known "shape" information for RegisterStatistics('flops') to calculate the statistics. It is suggested to pass in -run_meta_path if shape is only known during runtime. tfprof can fill in the missing shape with the runtime shape information from RunMetadata.

As for RunMetadata, there is a tf.RunMetadata() op in tensorflow, that should be what you need. Usually you pass it to a sess.run() op.

A solution for this is to pass RunMetadata during runtime, when all the shapes will be defined.

# Generate the RunMetadata that contains the memory and timing information.
#
# Note: When run on GPU, a kernel is first scheduled (enqueued) and then
#       executed asynchronously. tfprof only tracks the execution time.
#
run_metadata = tf.RunMetadata()
with tf.Session() as sess:
  _ = sess.run(train_op,
               options=tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE),
               run_metadata=run_metadata)

Hope this helps.