Python NLP Intent Identification

Question

I am novice in Python and NLP, and my problem is how to finding out Intent of given questions, for example I have sets of questions and answers like this :

question:What is NLP; answer: NLP stands for Natural Language Processing

I did some basic POS tagger on given questions in above question I get entety [NLP] I also did String Matching using this algo.

Basically I faced following issues :

1. If user ask what is NLP then it will return exact answers
2. If user ask meaning of NLP then it fail
3. If user ask Definition of NLP then it fail
4. If user ask What is Natural Language Processing then it fail

So how I should identify user intent of given questions because in my case String matching or pattern matching not works.

Answer 1

You can do intent identification with DeepPavlov, it supports multi-label classification. More information can be found in http://docs.deeppavlov.ai/en/master/components/classifiers.html The demo page https://demo.ipavlov.ai

Answer 2

you can use spacy for training a custom parser for chat intent semantics.

spaCy's parser component can be used to trained to predict any type of tree structure over your input text. You can also predict trees over whole documents or chat logs, with connections between the sentence-roots used to annotate discourse structure.

for example: "show me the best hotel in berlin"

('show', 'ROOT', 'show')
('best', 'QUALITY', 'hotel') --> hotel with QUALITY best
('hotel', 'PLACE', 'show') --> show PLACE hotel
('berlin', 'LOCATION', 'hotel') --> hotel with LOCATION berlin

To train the model you need data in this format:

# training data: texts, heads and dependency labels
# for no relation, we simply chose an arbitrary dependency label, e.g. '-'
TRAIN_DATA = [
    ("find a cafe with great wifi", {
        'heads': [0, 2, 0, 5, 5, 2],  # index of token head
        'deps': ['ROOT', '-', 'PLACE', '-', 'QUALITY', 'ATTRIBUTE']
    }),
    ("find a hotel near the beach", {
        'heads': [0, 2, 0, 5, 5, 2],
        'deps': ['ROOT', '-', 'PLACE', 'QUALITY', '-', 'ATTRIBUTE']
    })]

TEST_DATA:
input : show me the best hotel in berlin
output: [
      ('show', 'ROOT', 'show'),
      ('best', 'QUALITY', 'hotel'),
      ('hotel', 'PLACE', 'show'),
      ('berlin', 'LOCATION', 'hotel')
    ]

for more details Please check the below link. https://spacy.io/usage/examples#intent-parser