Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Unable to load the spacy model 'en_core_web_lg' on Google colab

I am using spacy in google colab to build an NER model for which I have downloaded the spaCy 'en_core_web_lg' model using

import spacy.cli
spacy.cli.download("en_core_web_lg")

and I get a message saying

✔ Download and installation successful
You can now load the model via spacy.load('en_core_web_lg')

However then when i try to load the model

nlp = spacy.load('en_core_web_lg')

the following error is printed:

OSError: [E050] Can't find model 'en_core_web_lg'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.

Could anyone help me with this problem?

like image 804
Jithin P James Avatar asked Jul 08 '19 02:07

Jithin P James


People also ask

How do you load a spaCy model in Colab?

use the keyboard shortcut Ctrl+M .

What is En_core_web_lg?

The model (en_core_web_lg) is the largest English model of spaCy with size 788 MB. There are smaller models in English and some other models for other languages (English, German, French, Spanish, Portuguese, Italian, Dutch, Greek). Step-3: Import Library and Load the Model.


3 Answers

Running

import spacy.cli
spacy.cli.download("en_core_web_lg")
nlp = spacy.load("en_core_web_lg")

shouldn't yield any errors anymore with recent spaCy versions.

If running the code still gives errors, you should be all set with running in one cell (takes a while, but gives you visual feedback about progress, differently from spacy.cli)

!python -m spacy download en_core_web_lg

Then, *** restart the colab runtime *** via

  • the colab menu Runtime > Restart runtime, or
  • use the keyboard shortcut Ctrl+M .

After that, executing

import spacy
nlp = spacy.load('en_core_web_lg')

should work flawlessly.

like image 200
Davide Fiocco Avatar answered Oct 01 '22 20:10

Davide Fiocco


In Google Colab Notebooks, you should import the model as a package.

However you download and install the model:

!pip install <model_s3_url> # tar.gz file e.g. from release notes like https://github.com/explosion/spacy-models/releases//tag/en_core_web_lg-2.3.1
!pip install en_core_web_lg
import spacy

you don't have permission in Colab to load the model with normal spacy usage:

nlp = spacy.load("en_core_web_lg") # not via packages
nlp = spacy.load("/path/to/en_core_web_lg") #not via paths
nlp = spacy.load("en") # nor via shortcut links
spacy.load()

Instead, import the model and load it directly:

import en_core_web_lg
nlp = en_core_web_lg.load()

Then use as directed:

doc = nlp("This is a sentence. Soon, it will be knowledge.")
like image 20
Briggsly Avatar answered Oct 01 '22 21:10

Briggsly


It seems the best answer is on this thread: How to install models/download packages on Google Colab?

import spacy.cli
spacy.cli.download("en_core_web_lg")
import en_core_web_lg
nlp = en_core_web_lg.load()
like image 26
GoPackGo Avatar answered Oct 01 '22 21:10

GoPackGo