Install Numpy Requirement in a Dockerfile. Results in error

Question

I am attempting to install a numpy dependancy inside a docker container. (My code heavily uses it). On building the container the numpy library simply does not install and the build fails. This is on OS raspbian-buster/stretch. This does however work when building the container on MAC OS.

I suspect some kind of python related issue, but can not for the life of me figure out how to make it work.

I should point out that removing the pip install numpy from the requirements file and using it in its own RUN statement in the dockerfile does not solve the issue.

The Dockerfile:

FROM python:3.6
ENV PYTHONUNBUFFERED 1
ENV APP /app
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN mkdir $APP
WORKDIR $APP
ADD requirements.txt .
RUN pip install -r requirements.txt
COPY . .

The requirements.txt contains all the project requirements, amounf which is numpy.

Step 6/15 : RUN pip install numpy==1.14.3
 ---> Running in 266a2132b078
Collecting numpy==1.14.3
  Downloading https://files.pythonhosted.org/packages/b0/2b/497c2bb7c660b2606d4a96e2035e92554429e139c6c71cdff67af66b58d2/numpy-1.14.3.zip (4.9MB)
Building wheels for collected packages: numpy
  Building wheel for numpy (setup.py): started
  Building wheel for numpy (setup.py): still running...
  Building wheel for numpy (setup.py): still running...

EDIT:

So after the comment by skybunk and the suggestion to head to official docs, some more debugging on my part, the solution wound up being pretty simple. Thanks skybunk to you go all the glory. Yay.

Solution:

Use alpine and install python install package dependencies, upgrade pip before doing a pip install requirements.

This is my edited Dockerfile - working obviously...

FROM python:3.6-alpine3.7

RUN apk add --no-cache --update \
    python3 python3-dev gcc \
    gfortran musl-dev \
    libffi-dev openssl-dev

RUN pip install --upgrade pip

ENV PYTHONUNBUFFERED 1
ENV APP /app

RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN mkdir $APP
WORKDIR $APP

ADD requirements.txt .
RUN pip install -r requirements.txt

COPY . .

Answer 1

To use Numpy on python3 here, we first head over to the official documentation to find what dependencies are required to build Numpy.

Mainly these 5 packages + their dependencies must be installed:

1. Python3 - 70 mb
2. Python3-dev - 25 mb
3. gfortran - 20 mb
4. gcc - 70 mb
5. musl-dev -10 mb (used for tracking unexpected behaviour/debugging)

An POC setup would look something like this -

Dockerfile:

FROM gliderlabs/alpine
ADD repositories.txt /etc/apk/repositories

RUN apk add --no-cache --update \
    python3 python3-dev gcc \
    gfortran musl-dev

ADD requirements-pip.txt .
RUN pip3 install --upgrade pip setuptools && \
    pip3 install -r requirements-pip.txt

ADD . /app
WORKDIR /app
ENV PYTHONPATH=/app/
ENTRYPOINT python3 testscript.py

repositories.txt

http://dl-5.alpinelinux.org/alpine/v3.4/main

requirements-pip.txt

numpy

testscript.py

import numpy as np

def random_array(a, b):
    return np.random.random((a, b))

a = random_array(2,2)
b = random_array(2,2)
print(np.dot(a,b))

To run this - clone alpine, build it using "docker build -t gliderlabs/alpine ."

Build and Run your Dockerfile

docker build -t minidocker .
docker run minidocker

Output should be something like this-

[[ 0.03573961 0.45351115]
[ 0.28302967 0.62914049]]

Here's the git link, if you want to test it out

Answer 2

From the error logs, it does not seem that it is from numpy. but you can install numpy before the requirment.txt and verify if it's working.

FROM python:3.6
RUN pip install numpy==1.14.3

Build

docker build -t numpy .

Run and Test

docker run numpy bash -c "echo import numpy as np > test.py ; python test.py"

So you will see no error on import.

or You can try numpy as an alpine package

FROM python:3-alpine3.9
RUN apk add --no-cache py3-numpy

Or better to post the requirement.txt.