Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Get phonemes from any word in Python NLTK or other modules?

Python NLTK has cmudict that spits out phonemes of recognized words. For example 'see' -> [u'S', u'IY1'], but for words that are not recognized it gives an error. For example 'seasee' -> error.

import nltk

arpabet = nltk.corpus.cmudict.dict()

for word in ('s', 'see', 'sea', 'compute', 'comput', 'seesea'):
    try:
        print arpabet[word][0]
    except Exception as e:
        print e

#Output
[u'EH1', u'S']
[u'S', u'IY1']
[u'S', u'IY1']
[u'K', u'AH0', u'M', u'P', u'Y', u'UW1', u'T']
'comput'
'seesea'

Is any there any module that doesn't have that limitation but able to find/guess phonemes of any real or made-up words?

If there is none, is there any way I can program it out? I am thinking about doing loops to test increasing portion of the word. For example in 'seasee', the first loop takes "s", next loop takes 'se', and third takes 'sea'... etc and run the cmudict. Though the problem is I don't know how to signal it's the right phoneme to consider. For example, both 's' and 'sea' in 'seasee' will output some valid phonemes.

Working progress:

import nltk

arpabet = nltk.corpus.cmudict.dict()

for word in ('s', 'see', 'sea', 'compute', 'comput', 'seesea', 'darfasasawwa'):
    try:
        phone = arpabet[word][0]
    except:
        try:
            counter = 0
            for i in word:
                substring = word[0:1+counter]
                counter += 1
                try:
                    print substring, arpabet[substring][0]
                except Exception as e:
                    print e
        except Exception as e:
            print e

#Output
c [u'S', u'IY1']
co [u'K', u'OW1']
com [u'K', u'AA1', u'M']
comp [u'K', u'AA1', u'M', u'P']
compu [u'K', u'AA1', u'M', u'P', u'Y', u'UW0']
comput 'comput'
s [u'EH1', u'S']
se [u'S', u'AW2', u'TH', u'IY1', u'S', u'T']
see [u'S', u'IY1']
sees [u'S', u'IY1', u'Z']
seese [u'S', u'IY1', u'Z']
seesea 'seesea'
d [u'D', u'IY1']
da [u'D', u'AA1']
dar [u'D', u'AA1', u'R']
darf 'darf'
darfa 'darfa'
darfas 'darfas'
darfasa 'darfasa'
darfasas 'darfasas'
darfasasa 'darfasasa'
darfasasaw 'darfasasaw'
darfasasaww 'darfasasaww'
darfasasawwa 'darfasasawwa'
like image 598
KubiK888 Avatar asked Nov 12 '15 07:11

KubiK888


1 Answers

I encountered the same issue, and I solved it by partitioning unknown recursively (see wordbreak)

import nltk
from functools import lru_cache
from itertools import product as iterprod

try:
    arpabet = nltk.corpus.cmudict.dict()
except LookupError:
    nltk.download('cmudict')
    arpabet = nltk.corpus.cmudict.dict()

@lru_cache()
def wordbreak(s):
    s = s.lower()
    if s in arpabet:
        return arpabet[s]
    middle = len(s)/2
    partition = sorted(list(range(len(s))), key=lambda x: (x-middle)**2-x)
    for i in partition:
        pre, suf = (s[:i], s[i:])
        if pre in arpabet and wordbreak(suf) is not None:
            return [x+y for x,y in iterprod(arpabet[pre], wordbreak(suf))]
    return None
like image 103
Uri Goren Avatar answered Sep 27 '22 16:09

Uri Goren