Extract emoticons from a text

Question

I need to extract text emoticons from a text using Python and I've been looking for some solutions to do this but most of them like this or this only cover simple emoticons. I need to parse all of them.

Currently I'm using a list of emoticons that I iterate for every text that I have process but this is so inefficient. Do you know a better solution? Maybe a Python library that can handle this problem?

Answer 1

One of most efficient solution is to use Aho–Corasick string matching algorithm and is nontrivial algorithm designed for this kind of problem. (search of multiple predefined strings in unknown text)

There is package available for this.
https://pypi.python.org/pypi/ahocorasick/0.9
https://hkn.eecs.berkeley.edu/~dyoo/python/ahocorasick/

Edit: There are also more recent packages available (haven tried any of them) https://pypi.python.org/pypi/pyahocorasick/1.0.0

Extra:
I have made some performance test with pyahocorasick and it is faster than python re when searching for more than 1 word in dict (2 or more).

Here it is code:

import re, ahocorasick,random,time

# search N words from dict
N=3

#file from http://norvig.com/big.txt
with open("big.txt","r") as f:
    text = f.read()

words = set(re.findall('[a-z]+', text.lower())) 
search_words = random.sample([w for w in words],N)

A = ahocorasick.Automaton()
for i,w in enumerate(search_words):
    A.add_word(w, (i, w))

A.make_automaton()
#test time for ahocorasic
start = time.time()
print("ah matches",sum(1 for i in A.iter(text))) 
print("aho done in ", time.time() - start)


exp = re.compile('|'.join(search_words))
#test time for re
start = time.time()
m = exp.findall(text)
print("re matches",sum(1 for _ in m))
print("re done in ",time.time()-start)