Logo Questions Linux Laravel Mysql Ubuntu Git Menu
HTML CSS JAVASCRIPT SQL PYTHON PHP BOOTSTRAP JAVA JQUERY R React Kotlin
 

Algorithm for generating a 'top list' using word frequency

Tags:

algorithm

text-processing

frequency

phrases

I have a big collection of human generated content. I want to find the words or phrases that occur most often. What is an efficient way to do this?

like image 312
Ryan Avatar asked Dec 29 '22 18:12

Ryan

1 Answers

Don't reinvent the wheel. Use a full text search engine such as Lucene.

like image 126
hobodave Avatar answered Jan 01 '23 08:01

hobodave
Sign in to Comment

Related questions

What is the complexity of empty for loop?
Skip multiple iterations in a loop
Time complexity of nested for loop
Find depth of object in iterative way
Efficient sum of expensive rolling window products
grouping of elements of same type together in an array in O(1) Space and O(n) time
Why does std::list have remove and remove_if functions
Function for calculating the vowels in a string
What data structure will be better fit - heap or sorted array?
Determine which color (red, blue, green or other) would be visible for a given RGB value combination?
Split array to sub-arrays 5 times while keeping unique pairs in all sub-arrays
Maximum length of the subsequence such that the bitwise XOR of each consecutive element is is k
Find consecutive range in a given list with limit
Algorithm to score similarness of sets of numbers
Pronouncable passwords?
Merging Treenodes
execution time calculation

Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!