Tensorflow LSTM throws ValueError: Shape () must have rank at least 2

Question

When attempting to run, the following exception (ValueError) is thrown

ValueError: Shape () must have rank at least 2

This is being thrown against the following line:

states_series, current_state = tf.contrib.rnn.static_rnn(cell, inputs_series, init_state)

Where cell is defined here:

cell = tf.contrib.rnn.BasicLSTMCell(state_size, state_is_tuple=True)

Looking at the rules for RNN and Tesor_shape, I can see that this is some kind of Tensor dimension shape issue. From what I can tell, it's failing to see BasicLSTMCell as a rank 2 matrix?

Full Error:

/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 /Users/glennhealy/PycharmProjects/firstRNNTest/LSTM-RNN.py
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
  return f(*args, **kwds)
Traceback (most recent call last):
  File "/Users/glennhealy/PycharmProjects/firstRNNTest/LSTM-RNN.py", line 42, in <module>
    states_series, current_state = tf.contrib.rnn.static_rnn(cell, inputs_series, init_state)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py", line 1181, in static_rnn
    input_shape = first_input.get_shape().with_rank_at_least(2)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/framework/tensor_shape.py", line 670, in with_rank_at_least
    raise ValueError("Shape %s must have rank at least %d" % (self, rank))
ValueError: Shape () must have rank at least 2

Process finished with exit code 1

The code:

state_size = 4
cell = tf.contrib.rnn.BasicLSTMCell(state_size, state_is_tuple=True)
states_series, current_state = tf.contrib.rnn.static_rnn(cell, inputs_series, init_state)

Tensorflow 1.2.1 Python 3.6 NumPy

Updating with further information:

Considering the advice given by @Maxim I can see that the issue is my input_series and this is causing the shape issue, however, I can't seem to get my head around his advice.

Some more information to help with things see if I can understand how to fix this:

Would the following be a replacement for my placeholders of BatchY and BatchX??

X = tf.placeholder(dtype=tf.float32, shape=[None, n_steps, n_inputs])
X_seqs = tf.unstack(tf.transpose(X, perm=[1, 0, 2]))
basic_cell = tf.nn.rnn_cell.BasicLSTMCell(num_units=n_neurons)
output_seqs, states = tf.nn.static_rnn(basic_cell, X_seqs,         dtype=tf.float32)

So, do I have to make changes to the following to reflect the syntax of the following?

batchX_placeholder = tf.placeholder(tf.int32, [batch_size,      truncated_backprop_length])
batchY_placeholder = tf.placeholder(tf.float32, [batch_size,    truncated_backprop_length])

#unpacking the columns:
labels_series = tf.unstack(batchY_placeholder, axis=1)
inputs_series = tf.split(1, truncated_backprop_length, batchX_placeholder)

#Forward pass
cell = tf.contrib.rnn.BasicLSTMCell(state_size, state_is_tuple=True)
states_series, current_state = tf.contrib.rnn.static_rnn(cell, inputs_series, init_state)

losses = [tf.nn.sparse_softmax_cross_entropy_with_logits(logits, labels) for logits, labels in zip(logits_series,labels_series)]
total_loss = tf.reduce_mean(losses)

Answer 1

Yes, the problem is with inputs_series. According to the error, it's a tensor with shape (), i.e., just a number.

From tf.nn.static_rnn documentation:

inputs: A length T list of inputs, each a Tensor of shape [batch_size, input_size], or a nested tuple of such elements.

In most cases, you want inputs to be [seq_length, None, input_size], where:

• seq_length is the sequence length, or the number of LSTM cells.
• None stands for the batch size (any).
• input_size is the number of features per cell.

So make sure your placeholders (and thus inputs_series, which is transformed from them) has the appropriate shape. Example:

X = tf.placeholder(dtype=tf.float32, shape=[None, n_steps, n_inputs])
X_seqs = tf.unstack(tf.transpose(X, perm=[1, 0, 2]))
basic_cell = tf.nn.rnn_cell.BasicLSTMCell(num_units=n_neurons)
output_seqs, states = tf.nn.static_rnn(basic_cell, X_seqs, dtype=tf.float32)

Update:

This is the wrong way to split the tensor:

# WRONG!
inputs_series = tf.split(1, truncated_backprop_length, batchX_placeholder)

You should do it like this (note the order of arguments):

inputs_series = tf.split(batchX_placeholder, truncated_backprop_length, axis=1)