Using tf.unpack() when first dimension of Variable is None

Question

I'm feeding in a dynamic shaped Tensor using:

x = tf.placeholder(tf.int32, shape=[None, vector_size])

I need to turn this into a list of Tensors that have shape=[1, vector_size] using x_list = tf.unpack(x, 0)

But it raises a ValueError because the length of the first dimension is not known i.e. it's None.

I've been trying to get around this by using another tf.placeholder to dynamically supply the shape of x but the parameter shape cannot be a Tensor.

How can I use tf.unpack() in this situation?

Or is there another function that can also turn the variable that I feed in into a list of Tensors?

Thanks in advance.

Answer 1

I don't think you can unpack a tensor with the argument num unspecified and non-inferrable. As their documentation says:

Raises ValueError if num is unspecified and cannot be inferred.

It has something to do with how TensorFlow's internal design for operations like unpack. In this other tread, Yaroslav Bulatov explained

Operations like unpack compile into "tensor-in/tensor-out" ops during graph construction time.

Hence TensorFlow needs to know the specific value of num to pass compiling.

Although, I'd try to get around this by using TensorArray. (see the following code for illustration).

import tensorflow as tf
import numpy as np
sess = tf.InteractiveSession()
# assume vector_size=2 for simplicity
x = tf.placeholder(tf.int32, shape=[None, 2])
TensorArr = tf.TensorArray(tf.int32, 1, dynamic_size=True, infer_shape=False)
x_array = TensorArr.unpack(x)

TensorArray is a class for wrapping dynamically sized arrays of Tensors. When initialize a TensorArray object in this application, TensorArr = tf.TensorArray(tf.int32, 1, dynamic_size=True, infer_shape=False), set dynamic_size=True and infer_shape=False since the shape of placeholder x is only partly defined.

To access each unpacked element:

# access the first element
x_elem0 = x_array.read(0)
# access the last element
last_idx = tf.placeholder(tf.int32)
x_last_elem = x_array.read(last_idx)

Then at evaluation time:

# generate random numpy array
dim0 = 4
x_np = np.random.randint(0, 25, size=[dim0, 2])
print x_np
# output of print x_np
[[17 15] 
[17 19]
[ 3  0]
[ 4 13]]

feed_dict = {x : x_np, last_idx : dim0-1} #python 0 based indexing
x_elem0.eval(feed_dict=feed_dict)
array([17, 15], dtype=int32) #output of x_elem0.eval(feed_dict)

x_last_elem.eval(feed_dict=feed_dict)
array([ 4, 13], dtype=int32) #output of x_last_elem.eval(feed_dict)
sess.close()

Note that when trying to access each unpacked element, if the index value is out of bound, you'd be able to pass the compiling but you'll get an error during runtime suggesting index out of bound. Additionally, the shape of the unpacked tensor would be TensorShape(None), since the shape of x is only partially determined until being evaluated.

Answer 2

Probably tf.dynamic_partition may help, but it requires static number of output tensors. If you can establish a maximum number of tensors then you can use it.

import tensorflow as tf
import numpy as np

x = tf.placeholder(tf.int32, shape=[None, 2])
data = np.random.randint(10, size=(10,2))

parts = range(len(data))
out = tf.dynamic_partition(x, parts, 20)

sess = tf.Session()
print 'out tensors:\n', out
print
print 'input data:\n', data
print
print 'sess.run result:\n', sess.run(out, {x: data})

This outputs the following:

out tensors:
[<tf.Tensor 'DynamicPartition:0' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:1' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:2' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:3' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:4' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:5' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:6' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:7' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:8' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:9' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:10' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:11' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:12' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:13' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:14' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:15' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:16' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:17' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:18' shape=(?, 2) dtype=int32>,
 <tf.Tensor 'DynamicPartition:19' shape=(?, 2) dtype=int32>]
input data:
[[7 6]
 [5 1]
 [4 6]
 [4 8]
 [4 9]
 [0 9]
 [9 6]
 [7 6]
 [0 5]
 [9 7]]

sess.run result:
[array([[7, 3]], dtype=int32),
 array([[0, 5]], dtype=int32),
 array([[2, 3]], dtype=int32),
 array([[2, 6]], dtype=int32),
 array([[7, 9]], dtype=int32),
 array([[8, 2]], dtype=int32),
 array([[1, 5]], dtype=int32),
 array([[3, 7]], dtype=int32),
 array([[6, 7]], dtype=int32),
 array([[8, 1]], dtype=int32),
 array([], shape=(0, 2), dtype=int32),
 array([], shape=(0, 2), dtype=int32),
 array([], shape=(0, 2), dtype=int32),
 array([], shape=(0, 2), dtype=int32),
 array([], shape=(0, 2), dtype=int32),
 array([], shape=(0, 2), dtype=int32),
 array([], shape=(0, 2), dtype=int32),
 array([], shape=(0, 2), dtype=int32),
 array([], shape=(0, 2), dtype=int32),
 array([], shape=(0, 2), dtype=int32)]