Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is difference between en_core_web_sm, en_core_web_md and en_core_web_lg model of spacy?

Tags:

python

spacy

I installed spacy on my system and I want to parse/extract person name, organization for english. But I saw here, there is 4 model for english. And there is model versioning. I didn't get which model is large and which I have to choose for development?

like image 691
Anil Jagtap Avatar asked May 23 '18 11:05

Anil Jagtap


People also ask

What is the difference between En_core_web_lg and En_core_web_sm?

All that while en_core_web_lg is 79 times larger, hence loads a lot more slowly. What I recommend is using the en_core_web_sm while developing and then switching to a larger model in production. You can easily switch just by changing the model you load.

What is En_core_web_sm spaCy?

For example, en_core_web_sm is a small English pipeline trained on written web text (blogs, news, comments), that includes vocabulary, syntax and entities.

What is En_core_web_lg?

The model (en_core_web_lg) is the largest English model of spaCy with size 788 MB. There are smaller models in English and some other models for other languages (English, German, French, Spanish, Portuguese, Italian, Dutch, Greek). Step-3: Import Library and Load the Model.

Which model does spaCy use?

EDIT Feb 2021: spaCy version 3 now uses the Transformer architecture as its deep learning model.


1 Answers

sm/md/lg refer to the sizes of the models (small, medium, large respectively).

As it says on the models page you linked to,

Model differences are mostly statistical. In general, we do expect larger models to be "better" and more accurate overall. Ultimately, it depends on your use case and requirements. We recommend starting with the default models (marked with a star below).

FWIW, the sm model is the default (as alluded to above)

like image 76
AKX Avatar answered Sep 28 '22 03:09

AKX