Menu
I need to implement a bidirectional LSTM network with a CRF layer at the end. Specifically the model presented in this paper, and train it.
http://www.aclweb.org/anthology/P15-1109
I want to implement it in Python preferably. Can anyone present some libraries or sample code as to how this can be done. I looked at PyBrain but couldn't really understand it.
I'm also open to tool-kits in other programming languages.
885
The LSTM-CRF is a hybrid graphical model which achieves state-of-the-art performance in supervised sequence labeling tasks. Collecting labeled data consumes lots of human resources and time. Thus, we want to improve the per- formance of LSTM-CRF by semi-supervised learning.
Bidirectional long-short term memory(bi-lstm) is the process of making any neural network o have the sequence information in both directions backwards (future to past) or forward(past to future). In bidirectional, our input flows in two directions, making a bi-lstm different from the regular LSTM.
The results show that additional training of data and thus BiLSTM-based modeling offers better predictions than regular LSTM-based models. More specifically, it was observed that BiLSTM models provide better predictions compared to ARIMA and LSTM models.
The results show that the bidirectional approach slightly enhances the recognition quality over the unidirectional approach. However, the bidirectional approach spends more time during the training, which may hinder its applicability on large datasets.
Here is an implementation of a bi-directional LSTM + CRF Network in TensorFlow: https://github.com/Franck-Dernoncourt/NeuroNER (works on Linux/Mac/Windows).
It gives state-of-the-art results on named-entity recognition datasets.
ANN architecture (it also uses character embeddings):
As viewed in TensorBoard:
You can also visualize the word embeddings:
154
There's this implementation by Guillaume Lample from the paper "Neural Architectures for Named Entity Recognition" that you can use for starter.
22
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
