How to do text classification with DeepPavlov

Question

I am interested in doing text classification with DeepPavlov chatbot framework.

The problem is I don't have enough training data. Ideally, I would like to do text classification with just few samples for each class.

Answer 1

You should check out DeepPavlov's autoFAQ models. There models were specifically developed to be effective when training data is limited.

There are few models at your disposal

• tf-idf based models

• fastText models

• and mix of both

Change the dataset source in the configuration file and train the model by running

python -m deeppavlov train tfidf_logreg_en_faq

You can interact with the trained model either via a command line

python -m deeppavlov interact tfidf_logreg_en_faq -d

or via the Python code

from deeppavlov.core.commands.infer import build_model
faq = build_model("tfidf_logreg_en_faq", load_trained = True, download = True)
a = faq(["I need help"])
a

You can find all required code snippets in the colab notebook