Python, ImportError: cannot import name AbstractLazySequence

Question

I am using nltk but the problem I am facing does not seem to be related to nltk specifically. I have a module named util.tokenize inside which there are some classes and I have the following first line:

util/tokenizer.py

from nltk.tokenize.regexp import RegexpTokenizer
...

class SentTokenizer(object):
    def __init__(self, stem=False, pattern='[^\w\-\']+'):
        self.alg = RegexpTokenizer(pattern, gaps=True)

    def __call__(self, text):
        return self.alg.tokenize(text)
    ....

if __name__ == '__main__':
     s_t = SentTokenizer()
     s_t('blah blah')

When I call those classes from another module, say test.py everything seems to work, but running the tokenize.py module directly causes ImportError.

File "tokenize.py", line 1, in <module>
...      
File "Python27\lib\site-packages\nltk\corpus\reader\util.py", line 28, in <module>
        from nltk.util import AbstractLazySequence, LazySubsequence, LazyConcatenation, py25
    ImportError: cannot import name AbstractLazySequence

What could be the problem? Why it works when called from other modules?

test.py

from util.tokenize import SentTokenizer
s_t = SentTokenizer()
print s_t('blah blah')

Platform is Windows.

Answer 1

We determined that this was being caused by a namespace conflict with nltk.tokenize and the user's tokenize.py. After renaming tokenize.py, everything worked properly.