Pip install killed - out of memory - how to get around it?

Question

I'm trying to install spaCy, a library for natural language processing.

root@vps:~# python3 -m pip install spacy
Collecting spacy
  Using cached https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3bd29384247babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylinux1_x86_64.whl
Installing collected packages: spacy
Killed

After a while, it's killed. I've already looked for answers on Stack Overflow and tried, and it's due to low memory (1,2 Gb free on my VPS, can't extend). A quick tail -f /var/log/kern.log gives me Out of memory: Kill process 31729 (python3) score 558 or sacrifice child.

I've already tried adding --no-cache-dir, but it didn't help. Something seems missing when copying the files manually. Is there any way to install this package on a low-RAM server?

EDIT : Complete log for python3 -m pip install spacy -vvv

Converted retries value: Retry(total=5, connect=None, read=None, redirect=None)                                                                                                              -> Retry(total=Retry(total=5, connect=None, read=None, redirect=None), connect=N                                                                                                             one, read=None, redirect=None)
Converted retries value: Retry(total=5, connect=None, read=None, redirect=None)                                                                                                              -> Retry(total=Retry(total=5, connect=None, read=None, redirect=None), connect=N                                                                                                             one, read=None, redirect=None)
Collecting spacy
  1 location(s) to search for versions of spacy:
  * https://pypi.python.org/simple/spacy/
  Getting page https://pypi.python.org/simple/spacy/
  Looking up "https://pypi.python.org/simple/spacy/" in the cache
  Returning cached "301 Moved Permanently" response (ignoring date and etag info                                                                                                             rmation)
  Looking up "https://pypi.org/simple/spacy/" in the cache
  Current age based on date: 29458
  Freshness lifetime from max-age: 600
  Freshness lifetime from request max-age: 600
  Starting new HTTPS connection (1): pypi.org
  https://pypi.org:443 "GET /simple/spacy/ HTTP/1.1" 304 0
  Analyzing links from page https://pypi.org/simple/spacy/
    <!-- Found link, skipped linked (hundreds of lines) -->
    Found link https://files.pythonhosted.org/packages/a7/84/f35f44560f0ab9ef23c                                                                                                             4a334a513f2b6c8a95d7b3347ef68a0247da63c08/spacy-2.1.6.tar.gz#sha256=b708bb582597                                                                                                             6220de78ad7e20e827912ca95653738a812cc2e5d194e3060ae8 (from https://pypi.org/simp                                                                                                             le/spacy/) (requires-python:>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*), version: 2.1                                                                                                             .6
  Using version 2.1.6 (newest of versions: 0.31, 0.32, 0.33, 0.40, 0.51, 0.52, 0                                                                                                             .60, 0.61, 0.62, 0.63, 0.64, 0.65, 0.67, 0.68, 0.70, 0.80, 0.81, 0.82, 0.83, 0.8                                                                                                             4, 0.85, 0.86, 0.87, 0.88, 0.89, 0.90, 0.91, 0.92, 0.93, 0.94, 0.95, 0.97, 0.98,                                                                                                              0.99, 0.100.0, 0.100.1, 0.100.2, 0.100.3, 0.100.4, 0.100.5, 0.100.6, 0.100.7, 0                                                                                                             .101.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0, 1.1.1, 1.1.2, 1.2.0, 1.3.0, 1.                                                                                                             4.0, 1.5.0, 1.5.1, 1.6.0, 1.7.0, 1.7.1, 1.7.2, 1.7.3, 1.7.5, 1.8.0, 1.8.1, 1.8.2                                                                                                             , 1.9.0, 1.10.0, 1.10.1, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.0.4, 2.0.5, 2.0.6, 2.0.7,                                                                                                              2.0.8, 2.0.9, 2.0.10, 2.0.11, 2.0.12, 2.0.13, 2.0.15, 2.0.16, 2.0.17, 2.0.18, 2                                                                                                             .1.0, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.1.5, 2.1.6)
  Looking up "https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3bd29384247                                                                                                             babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylinux1_x86_6                                                                                                             4.whl" in the cache
  Current age based on date: 98439
  Freshness lifetime from max-age: 365000000
  The response is "fresh", returning cached response
  365000000 > 98439
  Using cached https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3bd2938424                                                                                                             7babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylinux1_x86_                                                                                                             64.whl
  Downloading from URL https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3b                                                                                                             d29384247babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylin                                                                                                             ux1_x86_64.whl#sha256=2441ad9e6195669c13f4035dd77c4206414a40ebb8543c69f3386d984a                                                                                                             bfb942 (from https://pypi.org/simple/spacy/) (requires-python:>=2.7,!=3.0.*,!=3.                                                                                                             1.*,!=3.2.*,!=3.3.*)
Requirement already satisfied: plac<1.0.0,>=0.9.6 in /usr/local/lib/python3.5/di                                                                                                             st-packages (from spacy)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.5/d                                                                                                             ist-packages (from spacy)
Requirement already satisfied: srsly<1.1.0,>=0.0.6 in /usr/local/lib/python3.5/d                                                                                                             ist-packages (from spacy)
Requirement already satisfied: blis<0.3.0,>=0.2.2 in /usr/local/lib/python3.5/di                                                                                                             st-packages (from spacy)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/local/lib/python3                                                                                                             .5/dist-packages (from spacy)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/pytho                                                                                                             n3.5/dist-packages (from spacy)
Requirement already satisfied: wasabi<1.1.0,>=0.2.0 in /usr/local/lib/python3.5/                                                                                                             dist-packages (from spacy)
Requirement already satisfied: thinc<7.1.0,>=7.0.8 in /usr/local/lib/python3.5/d                                                                                                             ist-packages (from spacy)
Requirement already satisfied: preshed<2.1.0,>=2.0.1 in /usr/local/lib/python3.5                                                                                                             /dist-packages (from spacy)
Requirement already satisfied: numpy>=1.15.0 in /usr/local/lib/python3.5/dist-pa                                                                                                             ckages (from spacy)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/l                                                                                                             ocal/lib/python3.5/dist-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.5/dist-p                                                                                                             ackages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.5/di                                                                                                             st-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.5                                                                                                             /dist-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in /usr/local/lib/python3.5/d                                                                                                             ist-packages (from thinc<7.1.0,>=7.0.8->spacy)
Installing collected packages: spacy
Killed

Answer 1

This has been reported already on github

https://github.com/explosion/spaCy/issues/3396

Below are some good thread on how to solve the issue

fork() failing with Out of memory error

Memory error while using pip install Matplotlib

out of memory issue in installing packages on Ubuntu server

https://chirale.org/2017/01/15/memory-error-on-pip-install-solved/

In Summary, below are the two options:

1. Try with --no-cache-dir, which unfortunately has not helped you

2. Increase the swap space

$ create swap file of 512 MB
dd if=/dev/zero of=/swapfile bs=1024 count=524288

$ modify permissions
chown root:root /swapfile
chmod 0600 /swapfile

$ setup swap area
mkswap /swapfile

$ turn swap on
swapon /swapfile