Backpropagation (through time) code in Tensorflow

Question

Where can I find the backpropagation (through time) code in Tensorflow (python API)? Or are other algorithms used?

For example, when I create a LSTM net.

Answer 1

All backpropagation in TensorFlow is implemented by automatically differentiating the operations in the forward pass of the network, and adding explicit operations for computing the gradient at each point in the network. The general implementation can be found in tf.gradients(), but the particular version used depends on how your LSTM is implemented:

• If the LSTM is implemented as an unrolled loop for a finite number of timesteps, the usual approach is truncated backpropagation through time, which uses the algorithm in tf.gradients() to build an unrolled backpropagation loop in the opposite direction.
• If the LSTM is implemented as a tf.while_loop(), it uses additional support for differentiating loops in control_flow_grad.py.