Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

fuzzy string matching with grep

I am trying to match rows in a file containing a string say ACTGGGTAAACTA. If I do

grep "ACTGGGTAAACTA" file 

It gives me rows which have exact matches. Is there a way to allow for certain number of mismatches (substitutions, insertions or deletions)? For example, I am looking for sequences

  1. Up to 3 allowed subtitutions like "AGTGGGTAACCAA" etc.

  2. Insertions/deletions (having a partial match like "ACTGGGAAAATAAACTA" or "ACTAAACTA")

like image 505
Ssank Avatar asked May 20 '15 16:05

Ssank


3 Answers

There's a Python library called fuzzysearch (that I wrote) which provides precisely the required functionality.

Here's some sample code that should work:

from fuzzysearch import find_near_matches

with open('path/to/file', 'r') as f:
    data = f.read()

# 1. search allowing up to 3 substitutions
matches = find_near_matches("ACTGGGTAAACTA", data, max_substitutions=3)

# 2. also allow insertions and deletions, i.e. allow an edit distance
#    a.k.a. Levenshtein distance of up to 3
matches = find_near_matches("ACTGGGTAAACTA", data, max_l_dist=3)
like image 162
taleinat Avatar answered Nov 04 '22 11:11

taleinat


You can use tre-agrep and specify the edit distance with the -E switch. For example if you have a file foo:

cat <<< EOF > foo
ACTGGGAAAATAAACTA
ACTAAACTA
ACTGGGTAAACTA
EOF

You can match every line with an edit distance of up to 9 like this:

tre-agrep -s -9 -w ACTGGGTAAACTA foo

Output:

4:ACTGGGAAAATAAACTA
4:ACTAAACTA
0:ACTGGGTAAACTA
like image 41
Thor Avatar answered Nov 04 '22 09:11

Thor


There used to be a tool called agrep for fuzzy regex matching, but it got abandoned.

http://en.wikipedia.org/wiki/Agrep has a bit of history and links to related tools.

https://github.com/Wikinaut/agrep looks like a revived open source release, but I have not tested it.

Failing that, see if you can find tre-agrep for your distro.

like image 5
tripleee Avatar answered Nov 04 '22 09:11

tripleee