Is there any tool or utility or perl/python script that can find longest repeated substrings in a large text file and print those patterns and the number of times each pattern occurs?
http://en.wikipedia.org/wiki/Longest_repeated_substring_problem:
The longest repeated substring problem is finding the longest substring of a string that occurs at least twice. This problem can be solved in linear time and space by building a suffix tree for the string, and finding the deepest internal node in the tree
Suffix trees in python (a little dated, though ..): http://hkn.eecs.berkeley.edu/~dyoo/python/suffix_trees/
Javascript implementation with further explaination: http://www.allisons.org/ll/AlgDS/Tree/Suffix/
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With