Multi-layered bidirectional_dynamic_rnn: incompatible with MultiRNNCell?

Question

I want to create a multi-layered bidirectional LSTM in Tensorflow. Currently my single-layered model looks like:

cell_fw = tf.contrib.rnn.LSTMCell(hidden_size)
cell_bw = tf.contrib.rnn.LSTMCell(hidden_size)

(self.out_fw, self.out_bw), _ = tf.nn.bidirectional_dynamic_rnn(cell_fw, cell_bw, input, ...)

In order to turn this into a multi-layered I suspect I can not simply wrap a few LSTMCells with MultiRNNCells like so:

multi_cell_fw = tf.contrib.rnn.MultiRNNCell([cell_fw] * num_layers, ...)

and feed them into the bidirectional_dynamic_rnn since both forward and backward LSTMs in each layer need the output of both the forward and backward directions of the preceding layer. Currently my solution is to create my bidirectional_dynamic_rnns in a loop, feeding in the concatenated output of LSTMs of the preceding layers.

However, it's not very clean and frankly I'm not sure if it's correct, though it does work on a toy dataset. Is there a better way that's comparably elegant to using something like MultiRNNCell?

I'm using Tensorflow API r1.0.

Answer 1

Just do:

multi_cell_fw = tf.contrib.rnn.MultiRNNCell([cell_fw for _ in range(num_layers)], ...)

That should work.