Trouble understanding tf.contrib.seq2seq.TrainingHelper

Question

I managed to build a sequence to sequence model in tensorflow using the tf.contrib.seq2seq classes in 1.1 version.

For know I use the TrainingHelper for training my model. But does this helper feed previously decoded values in the decoder for training or just the ground truth? If it doesn't how can I feed previously decoded value as input in the decoder instead of ground truth values ?

Answer 1

TrainingHelper feeds the ground truth at every step. If you want to use decoder outputs, you can use scheduled sampling [1]. Scheduled sampling is implemented in ScheduledEmbeddingTrainingHelper and ScheduledOutputTrainingHelper, so you can use one of the two (depending on your particular application) instead of TrainingHelper. See also this thread here: scheduled sampling in Tensorflow.

[1] https://arxiv.org/pdf/1506.03099.pdf