Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Use Python package installed with apk in Alpine Linux

I'd like to install some Python packages in Alpine Linux using apk. I use numpy as an example in the following.

Dockerfile

FROM python:3-alpine
RUN apk add --update py3-numpy

I build my Docker image

$ docker build -t python-numpy .
Sending build context to Docker daemon  2.048kB
Step 1/2 : FROM python:3-alpine
 ---> 930a7e894675
Step 2/2 : RUN apk add --update py3-numpy
 ---> Running in b30470090cde
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/x86_64/APKINDEX.tar.gz
(1/6) Installing libgcc (8.3.0-r0)
(2/6) Installing libquadmath (8.3.0-r0)
(3/6) Installing libgfortran (8.3.0-r0)
(4/6) Installing openblas (0.3.6-r0)
(5/6) Installing python3 (3.7.3-r0)
(6/6) Installing py3-numpy (1.16.4-r1)
Executing busybox-1.30.1-r2.trigger
OK: 113 MiB in 41 packages
Removing intermediate container b30470090cde
 ---> 5a82ffa67522
Successfully built 5a82ffa67522
Successfully tagged python-numpy:latest

run it and try to import the package in python

$ docker run -it --rm python-numpy python -c "import numpy"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'

However, it is not found. Running pip install numpy after apk add --update py3-numpy does not consider the apk package py3-numpy and downloads another version:

Collecting numpy
  Downloading https://files.pythonhosted.org/packages/da/32/1b8f2bb5fb50e4db68543eb85ce37b9fa6660cd05b58bddfafafa7ed62da/numpy-1.17.0.zip (6.5MB)
...

If I specify the same version as of py3-numpy (see output of docker build) in pip install numpy==1.16.4-r1, it leads to

Collecting numpy==1.16.4-r1
  ERROR: Could not find a version that satisfies the requirement numpy==1.16.4-r1 (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0b3, 1.11.0rc1, 1.11.0rc2, 1.11.0, 1.11.1rc1, 1.11.1, 1.11.2rc1, 1.11.2, 1.11.3, 1.12.0b1, 1.12.0rc1, 1.12.0rc2, 1.12.0, 1.12.1rc1, 1.12.1, 1.13.0rc1, 1.13.0rc2, 1.13.0, 1.13.1, 1.13.3, 1.14.0rc1, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0rc1, 1.15.0rc2, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0rc1, 1.16.0rc2, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.17.0rc1, 1.17.0rc2, 1.17.0)
ERROR: No matching distribution found for numpy==1.16.4-r1

What am I missing?

like image 895
maiermic Avatar asked Aug 09 '19 17:08

maiermic


People also ask

Where does apk install to Alpine?

You can install packages from a local disk (such as CDROM or a USB stick) or the internet archive location using apk command, the Alpine package manager for binary packages, instead of compiling them from the source. The list of repositories is stored in /etc/apk/repositories configuration file.

What is apk on Alpine Linux?

apk is the Alpine Package Keeper - the distribution's package manager. It is used to manage the packages (software and otherwise) of the system. It is the primary method for installing additional software, and is available in the apk-tools package.

Does Alpine have Python installed?

alpine does not have a python package. Instead, it ships a python3 package instead. The python installation does not include pip .

What is APK in Alpine Linux?

In Alpine Linux, APK, short for Alpine Package Keeper, is the package management tool. It retrieves packages and information about the packages from online repositories. The main repository contains packages that are directly supported and updated by the Alpine Linux core team.

How do I install a package in Alpine Linux?

Install packages in Alpine Linux To add or install a package, for example vim, in Alpine Linux, simply run: $ sudo apk add vim The above command will install vim and its dependencies.

What is APK package manager?

A brief introduction to apk package manager. Apk, stands for Alpine Package Keeper, is the default package manager for Alpine Linux. It is used to install, update, upgrade, search, list and remove packages on a running Alpine Linux system. Apk is the part of apk-tools package which comes pre-installed in all Alpine Linux versions.

What is the Alpine Linux main repository?

The main repository contains packages that are directly supported and updated by the Alpine Linux core team. In addition, the packages also come with official documentation and are available for all Alpine Linux releases. Packages from the main repository will always have replacements if they do not go past upstream.


2 Answers

The issue is that python:3-alpine has two Pythons: the one provided by Alpine, and an additional one added by the Python Docker image. Installing packages in one will not be reflected in the other.

Some options:

  1. Switch to just alpine base image, FROM alpine:3.10. Then you'll just have the Python installed via apk.
  2. Stop using Alpine, and switch to FROM python:3.7-slim-buster (my personal recommendation: https://pythonspeed.com/articles/base-image-python-docker-images/). This will allow you to pip install numpy without having to compile anything—binary wheels don't work on Alpine, but will work on the (Debian) Buster image.
like image 169
Itamar Turner-Trauring Avatar answered Nov 04 '22 19:11

Itamar Turner-Trauring


EDIT: This approach is not recommended, since the apk packages are for a different python version (see Itamar Turner-Trauring's answer).


apk installs python packages in /usr/lib/python3.7/site-packages. This path is not part of Python's sys.path (in Docker images). Hence, packages installed with apk are not found. The installation directory has to be added to the search path:

FROM python:3.7-alpine
RUN apk add --update py3-numpy
ENV PYTHONPATH /usr/lib/python3.7/site-packages

Note that there is no /usr/lib/python3/site-packages and even if you use python:3.6-alpine, apk add --update py3-numpy creates a directory /usr/lib/python3.7 and not /usr/lib/python3.6.


By the way, you have a similar issue if you are using python:3, but the installation directory is different

FROM python:3
RUN apt-get update && apt-get install -y python3-numpy
ENV PYTHONPATH /usr/lib/python3/dist-packages
like image 23
maiermic Avatar answered Nov 04 '22 20:11

maiermic