Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

NLTK - Download all nltk data except corpara from command line without Downloader UI

We can download all nltk data using:

> import nltk
> nltk.download('all')

Or specific data using:

> nltk.download('punkt')
> nltk.download('maxent_treebank_pos_tagger')

But I want to download all data except 'corpara' files, for example - all chunkers, grammers, models, stemmers, taggers, tokenizers, etc

is there any way to do so without Downloader UI? something like,

> nltk.download('all-taggers')
like image 707
RAVI Avatar asked Jun 25 '16 16:06

RAVI


People also ask

Where can I download NLTK data?

Download individual packages from https://www.nltk.org/nltk_data/ (see the “download” links). Unzip them to the appropriate subfolder.


1 Answers

List all corpora ids and set _status_cache[pkg.id] = 'installed'.

It will set status value for all corpora as 'installed' and corpora packages will be skipped when we use nltk.download().

Instead of downloading all corpora and models, if you're unsure of which corpora/package you need, use nltk.download('popular').

import nltk

dwlr = nltk.downloader.Downloader()

for pkg in dwlr.corpora():
    dwlr._status_cache[pkg.id] = 'installed'

dwlr.download('popular')

To download all packages of specific folder.

import nltk

dwlr = nltk.downloader.Downloader()

# chunkers, corpora, grammars, help, misc, 
# models, sentiment, stemmers, taggers, tokenizers
for pkg in dwlr.packages():
    if pkg.subdir== 'taggers':
        dwlr.download(pkg.id)
like image 142
RAVI Avatar answered Sep 25 '22 00:09

RAVI