Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Algorithm for Natural-Looking Sentence in English Language

Tags:

c#

.net

nlp

I'm building an application that does sentence checking. Do you know are there any DLLs out there that recognize sentences and their logic and organize sentences correctly? Like put words in a sentence into a correct sentence.

If it's not available, maybe you can suggest search terms that I can research.

like image 709
Jason Avatar asked Feb 21 '23 23:02

Jason


1 Answers

There are things called language model and n-gram. I'll try shortly explain what they are. Suppose you have a huge coolection of correct english sentences. Let's pick one of them:

The quick brown fox jumps over the lazy dog. Let's now look at all the pairs of words (called bigrams) in it: (the, quick), (quick, brown), (brown, fox), (fox, jumps) and so on... Having a huge collection of sentences we will have a huge number of bigrams. We now take unique ones and count their frequences (number of time we saw it in correct sentences). We now have, say
('the', quick) - 500
('quick', brown) - 53

Bigrams with their frequencies called a language model. It shows you how common a certain combination of words is.

So you can build all the possible sentences of your words an count a weight of each of them taking in account language model. A sentence with the max weight is going to be what you need.

Where to take bigrams and their frequencies? Well, google has it. You can use not just a pair of words, but triples and so on. It will allow you to build more human-like sentences.

like image 65
StuffHappens Avatar answered Mar 09 '23 00:03

StuffHappens