TensorFlow: simple recurrent neural network

Question

I've built some neural networks with TensorFlow, like basic MLPs and convolutional neural networks. Now I want to move on to recurrent neural networks. However, I'm not experienced in natural language processing. Therefore the TensorFlow NLP tutorials for RNNs are not easy to read for me (and not really interesting, too).

Basically I want to start off with something simple, not a LSTM.

How would one build a simple recurrent neural network, like an Elman network, in TensorFlow?

I were only able to find GRU- or LSTM RNN examples for TensorFlow, mostly for NLP. Does anyone know of some simple recurrent neural network tutorials or examples for TensorFlow?

This figure shows a basic Elman network, which is often simply called SRN (simple recurrent network):

Answer 1

One option is to use the built-in RNNCell located in tensorflow/python/ops/rnn_cell.py.

If you don't want to do that you can make your own RNN. The RNN will train using back-propagation through time. Try unrolling the network a fixed number of steps, e.g. consider input sequences of length ten. Then you can write a loop in python to do all of the matrix multiplications for each step of the network. Each time you can take the output from the previous step and concatenate it with the input to that step. It will not be too many lines of code to get this working.