I need to install a Python package in a custom Docker container that I'm building from the official 'ubuntu' Docker image, so I want to minimize how much space this uses. Python3 installs fine and runs, but for some reason, pip is not included.
So I installed via apt install python3-pip
, this works but it is a whopping 300 megs and takes a couple of minutes to install (apparently because it installs a sh*load of stuff to build binary packages from gcc etc).
Of course I could uninstall python3-pip from the image after installing the dependencies I want, and additionally use apt autoremove
to get rid of 299 megs. However this takes another minute.
So although the above works, it significantly increases the build time of my Docker image. So I tried to see if there was a way of installing the dependency without pip:
I tried downloading the dependency's .tar.gz
from PyPI, extracted, and tried python3 setup.py install
, but this gets me an odd error:
Traceback (most recent call last):
File "setup.py", line 59, in <module>
from distutils import log
ImportError: cannot import name 'log'
I thought perhaps I need to install setuptools, or upgrade distutils.
I tried to use get-pip.py
from the official site but that failed too:
Traceback (most recent call last):
File "get-pip.py", line 20890, in <module>
main()
File "get-pip.py", line 197, in main
bootstrap(tmpdir=tmpdir)
File "get-pip.py", line 82, in bootstrap
import pip._internal
File "/tmp/tmpjpa5gs_x/pip.zip/pip/_internal/__init__.py", line 40, in <module>
File "/tmp/tmpjpa5gs_x/pip.zip/pip/_internal/cli/autocompletion.py", line 8, in <module>
File "/tmp/tmpjpa5gs_x/pip.zip/pip/_internal/cli/main_parser.py", line 8, in <module>
File "/tmp/tmpjpa5gs_x/pip.zip/pip/_internal/cli/cmdoptions.py", line 17, in <module>
File "/tmp/tmpjpa5gs_x/pip.zip/pip/_internal/locations.py", line 10, in <module>
ImportError: cannot import name 'sysconfig'
which is very weird because if I start python3, import sysconfig
works fine.
I also tried apt install python-pyyaml
(the dependency I need in my Docker image) but that doesn't seem to exist.
So I'm out of options.
Most Python packages are now designed to be compatible with Python's pip package manager. But if you have a package that is not compatible with pip, you'll need manually install Python packages.
npm, Homebrew, Yarn, RequireJS, and Bower are the most popular alternatives and competitors to pip.
To install pip for Python 3 on Ubuntu 20.04 run the following commands as root or sudo user in your terminal: sudo apt update sudo apt install python3-pip. Copy. Copy. The command above will also install all the dependencies required for building Python modules.
As a package management system, pip lets you install packages of software written in the Python programming language. In this tutorial, we will explain what pip is, how it works, and the ways to install pip on Ubuntu using Python 2 and Python 3.
There is one more way to install packages in Ubuntu. It’s called Pip and you can use it to install Python-based applications. Pip stands for “Pip Installs Packages”. Pip is a command line based package management system. It is used to install and manage software written in Python language.
Ubuntu 18.04 has both Python 2 and Python 3 installed by default. And hence, you should install pip for both Python versions. Pip, by default, refers to the Python 2. Pip in Python 3 is referred by pip3. Note: I am using Ubuntu 18.04 in this tutorial.
I ran into a similar issue and wanted to give an alternative solution.
On Ubuntu 20.04 build-essential and python3-dev are recommended packages for python3-pip, therefore you can use the --no-install-recommends
option to skip them:
RUN apt update -y && \
apt install python3 python3-pip --no-install-recommends -y && \
apt clean
This took my image from 420MB to 165MB, and obviously the build time was also quicker.
Note: this will work fine for pure-Python packages, but you will likely need build-essential and python3-dev if you want to compile anything
The Debian and Ubuntu package for PyYAML is called python-yaml
, not python-pyyaml
.
sudo apt install python-yaml
or
sudo apt install python3-yaml
respectively.
(It seems to be common in Debian/Ubuntu package names to drop any additional "py" prefix that a Python package might have: it's apt install python-tz
instead of python-pytz
, too. They don't seem to like the py-redundancy.)
I have been installing pip
from the pip-get.py
in Docker (Ubuntu) containers for a few years without problem. For me it is the best way to not-get pip-out-of-date warnings or (at some point, some time ago) SSL related errors.
So the second part of your answer is close, but your python install seems a bit too minimal, you need sysconfig
as provided by python-distutils
. You can try this rather minimal Dockerfile
:
FROM ubuntu:latest
MAINTAINER Anthon
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update && apt-get install -y \
python3 \
python3-distutils \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# this gets you the latest pip
COPY pip/get-pip.py /tmp/get-pip.py
RUN python3 /tmp/get-pip.py
RUN pip3 install pyyaml
which I ran using this Makefile
:
doit: pip/get-pip.py
docker build .
pip/get-pip.py:
-@mkdir pip
curl https://bootstrap.pypa.io/get-pip.py -o pip/get-pip.py
(those need to be TAB characters on the indented lines) to make sure pip-get.py
is available from the context (you can of course download it from within the Dockerfile, but that is not necessary). That ends in a succesful PyYAML install, but it will be slow.
I recommend you start using ruamel.yaml (disclaimer: I am the author of that package), by changing the last line of the Dockerfile to read:
RUN pip3 install ruamel.yaml
Apart from many bugfixes in the original PyYAML code it is based upon, ruamel.yaml
supports YAML 1.2 and YAML 1.1 (replaced in 2009 and the version PyYAML supports), and installs the appropriate version from a .whl
file, so you'll have the fast C loader available in your container (PyYAML doesn't do that).
You can load a YAML file using the C-loader in ruamel.yaml
using:
from pathlib import Path
from ruamel.yaml import YAML
path = Path('yourfile.yaml')
yaml = YAML(typ='safe')
data = yaml.load(path)
When there is no apt install something
that is available for a python package, here is how to do it. Thanks to @Anthon and @digitalarbeiter as their answers provided important information to arrive at solution.
To install via a setup.py file (esp. useful in Ubuntu Docker container):
it was sufficient for me to
apt install python3-distutils
<download package, tar xvf, cd to folder>
python3 setup.py install
This method of installation only works for pure Python packages (should not not be a surprise), which means that Python packages that have non-pure Python dependencies may not install or, if they do, will have some functionality unavailable.
python3 -m distutils
worked; this implies that the builtin distutils, that comes with Python3 via apt install python3
, is not the full distutils; I did not know that, is this fact ever mentioned anywhere?To install pip without the gcc toolchain: it was sufficient for me to
apt install python3-distutils
wget https://bootstrap.pypa.io/get-pip.py
python3 get-pip.py
Then pip install pyyaml
completed. It seemed to install from a .tar.gz
so it is the pure Python implementation too. Not surprising. This technique is useful if a package is not installable via apt install python3-<package>
The above methods required only a few megs of disk space.
A couple other missing pieces of the puzzle for me were:
apt install python3-<something>
:
py<something>
on PyPI, the apt install is python3-<something>
NOT python3-py<something>
. Unfortunately apt search pyyaml
was no help here. apt search <something>
: I had sort of forgotten about it because bash on (desktop) Ubuntu automatically suggests the right package to download when a command is not found.
apt search yaml
yields 81 packages that provide YAML read/write in several languages (Python 2, Python 3, nodejs, Java, Go, Ruby, Erlang, Lua, Perl, C, C++, Clojure), linter, schema validator, etc. apt search yaml python3
showed the python3-yaml that I missed.apt search pyyaml
produces no results, eventhough the Source
and Homepage
fields of apt show python3-yaml
contain the word "pyyaml". I could not find a way to make search
include those fields.If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With