NLTK: Package Errors? punkt and pickle?

Question

Basically, I have no idea why I'm getting this error.

Just to have more than an image, here is a similar message in code format. As it is more recent, the answer of this thread has already been mentioned in the message:

Preprocessing raw texts ...

---------------------------------------------------------------------------

LookupError                               Traceback (most recent call last)

<ipython-input-38-263240bbee7e> in <module>()
----> 1 main()

7 frames

<ipython-input-32-62fa346501e8> in main()
     32     data = data.fillna('')  # only the comments has NaN's
     33     rws = data.abstract
---> 34     sentences, token_lists, idx_in = preprocess(rws, samp_size=samp_size)
     35     # Define the topic model object
     36     #tm = Topic_Model(k = 10), method = TFIDF)

<ipython-input-31-f75213289788> in preprocess(docs, samp_size)
     25     for i, idx in enumerate(samp):
     26         sentence = preprocess_sent(docs[idx])
---> 27         token_list = preprocess_word(sentence)
     28         if token_list:
     29             idx_in.append(idx)

<ipython-input-29-eddacbfa6443> in preprocess_word(s)
    179     if not s:
    180         return None
--> 181     w_list = word_tokenize(s)
    182     w_list = f_punct(w_list)
    183     w_list = f_noun(w_list)

/usr/local/lib/python3.7/dist-packages/nltk/tokenize/__init__.py in word_tokenize(text, language, preserve_line)
    126     :type preserver_line: bool
    127     """
--> 128     sentences = [text] if preserve_line else sent_tokenize(text, language)
    129     return [token for sent in sentences
    130             for token in _treebank_word_tokenizer.tokenize(sent)]

/usr/local/lib/python3.7/dist-packages/nltk/tokenize/__init__.py in sent_tokenize(text, language)
     92     :param language: the model name in the Punkt corpus
     93     """
---> 94     tokenizer = load('tokenizers/punkt/{0}.pickle'.format(language))
     95     return tokenizer.tokenize(text)
     96 

/usr/local/lib/python3.7/dist-packages/nltk/data.py in load(resource_url, format, cache, verbose, logic_parser, fstruct_reader, encoding)
    832 
    833     # Load the resource.
--> 834     opened_resource = _open(resource_url)
    835 
    836     if format == 'raw':

/usr/local/lib/python3.7/dist-packages/nltk/data.py in _open(resource_url)
    950 
    951     if protocol is None or protocol.lower() == 'nltk':
--> 952         return find(path_, path + ['']).open()
    953     elif protocol.lower() == 'file':
    954         # urllib might not use mode='rb', so handle this one ourselves:

/usr/local/lib/python3.7/dist-packages/nltk/data.py in find(resource_name, paths)
    671     sep = '*' * 70
    672     resource_not_found = '\n%s\n%s\n%s\n' % (sep, msg, sep)
--> 673     raise LookupError(resource_not_found)
    674 
    675 

LookupError: 
**********************************************************************
  Resource punkt not found.
  Please use the NLTK Downloader to obtain the resource:

  >>> import nltk
  >>> nltk.download('punkt')
  
  Searched in:
    - '/root/nltk_data'
    - '/usr/share/nltk_data'
    - '/usr/local/share/nltk_data'
    - '/usr/lib/nltk_data'
    - '/usr/local/lib/nltk_data'
    - '/usr/nltk_data'
    - '/usr/lib/nltk_data'
    - ''
**********************************************************************

Answer 1

Perform the following:

>>> import nltk
>>> nltk.download()

Then when you receive a window popup, select punkt under the identifier column which is locatedin the Module tab.

Answer 2

Do:

>>> import nltk
>>> nltk.download('punkt')
>>> from nltk import sent_tokenize

To download all dataset and models:

>>> nltk.download('all')

Ensure that you've the latest version of NLTK because it's always improving and constantly maintain:

$ pip install --upgrade nltk

---

Similar question on Windows/Linux but with the above code snippet don't help:

• error installing nltk supporting packages : nltk.download()
• Python nltk download and download_shell both freeze (hang) on punkt attempt
• Proxy error while downloading NLTK
• NLTK Wordnet Download Out of Date
• NLTK and Stopwords Fail #lookuperror
• https://askubuntu.com/questions/527388/python-nltk-on-ubuntu-12-04-lts-nltk-downloadbrown-results-in-html-error-40

Similar question on Windows:

• NLTK and Stopwords Fail #lookuperror

Similar question on Linux platform:

• downloading error using nltk.download()
• How do I download NLTK data?
• Failed loading english.pickle with nltk.data.load
• https://stackoverflow.com/questions/29206471/downloading-nltk-corpora-for-python-3-4
-

Similar question on OSX:

• nltk.download() hangs on OS X

Similar question but has some authorization/authentication errors:

• nltk download url authorization issue
• nltk.download() failed with the http error
• HTTP: Proxy Authentification Error for nltk.download()
-

Similar question that tries to install NLTK data outside of python interpreter:

• Installing nltk data in setup.py script