How to implement sentence-level log-likelihood in tensorflow?

Question

I want to implement the sentence-level log-likelihood as described in Collobert et al., p. 14.

To compute transition scores, I could use CRF, but I don't know how to integrate it in tensorflow. I thought about using tf.contrib.crf.CrfForwardRnnCell to compute transition scores, but this class returns a pair of [batch_size, num_tags] matrices values containing the new alpha values and not as I would expect one [batch_size, num_tags, num_tags] tensor.

Does anyone has an example how to use CRF in tensorflow? Thank you!

Answer 1

A good example of using contrib.crf in TensorFlow is given here: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/crf

It's worth noting that the SLL objective described in the paper Collobert et al. 2011 is slightly different than the CRF objective in that SLL lacks normalization (see Remark 4 on p. 16), but this shouldn't really matter in practice (I'd just use the CRF.)