Docker unable to install numpy, scipy, or gensim

Question

I am trying to build a Docker application that uses Python's gensim library, version 2.1.0, which is being installed via pip from a requirements.txt file.

However, Docker seems to have trouble installing numpy, scipy, and gensim. I googled the error messages and found other users who experienced the same problem, but in other environments. Many of their solutions do not seem to work in Docker.

The following is the error message:

<pre> Step 4 : RUN pip install -r requirements.txt
 ---> Running in a86d07e229d7
Collecting Flask==0.12 (from -r requirements.txt (line 1))
  Downloading Flask-0.12-py2.py3-none-any.whl (82kB)
Collecting requests==2.17.3 (from -r requirements.txt (line 2))
  Downloading requests-2.17.3-py2.py3-none-any.whl (87kB)
Collecting numpy==1.12.1 (from -r requirements.txt (line 3))
  Downloading numpy-1.12.1.zip (4.8MB)
Collecting nltk==3.2.2 (from -r requirements.txt (line 4))
  Downloading nltk-3.2.2.tar.gz (1.2MB)
Collecting scipy==0.19.0 (from -r requirements.txt (line 5))
  Downloading scipy-0.19.0.zip (15.3MB)
    Complete output from command python setup.py egg_info:
    /bin/sh: svnversion: not found
    /bin/sh: svnversion: not found
    non-existing path in 'numpy/distutils': 'site.cfg'
    Could not locate executable gfortran
    Could not locate executable f95
    Could not locate executable ifort
    Could not locate executable ifc
    Could not locate executable lf95
    Could not locate executable pgfortran
    Could not locate executable f90
    Could not locate executable f77
    Could not locate executable fort
    Could not locate executable efort
    Could not locate executable efc
    Could not locate executable g77
    Could not locate executable g95
    Could not locate executable pathf95
    don't know how to compile Fortran code on platform 'posix'
    Running from numpy source directory.
    /tmp/easy_install-ocgjhe9m/numpy-1.13.0/setup.py:367: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
      run_build = parse_setuppy_commands()
    /tmp/easy_install-ocgjhe9m/numpy-1.13.0/numpy/distutils/system_info.py:572: UserWarning:
        Atlas (http://math-atlas.sourceforge.net/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [atlas]) or by setting
        the ATLAS environment variable.
      self.calc_info()
    /tmp/easy_install-ocgjhe9m/numpy-1.13.0/numpy/distutils/system_info.py:572: UserWarning:
        Blas (http://www.netlib.org/blas/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [blas]) or by setting
        the BLAS environment variable.
      self.calc_info()
    /tmp/easy_install-ocgjhe9m/numpy-1.13.0/numpy/distutils/system_info.py:572: UserWarning:
        Blas (http://www.netlib.org/blas/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [blas_src]) or by setting
        the BLAS_SRC environment variable.
      self.calc_info()
    /tmp/easy_install-ocgjhe9m/numpy-1.13.0/numpy/distutils/system_info.py:572: UserWarning:
        Lapack (http://www.netlib.org/lapack/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [lapack]) or by setting
        the LAPACK environment variable.
      self.calc_info()
    /tmp/easy_install-ocgjhe9m/numpy-1.13.0/numpy/distutils/system_info.py:572: UserWarning:
        Lapack (http://www.netlib.org/lapack/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [lapack_src]) or by setting
        the LAPACK_SRC environment variable.
      self.calc_info()
    /usr/local/lib/python3.5/distutils/dist.py:261: UserWarning: Unknown distribution option: 'define_macros'
      warnings.warn(msg)
    Traceback (most recent call last):
      File "/usr/local/lib/python3.5/site-packages/setuptools/sandbox.py", line 158, in save_modules
        yield saved
      File "/usr/local/lib/python3.5/site-packages/setuptools/sandbox.py", line 199, in setup_context
        yield
      File "/usr/local/lib/python3.5/site-packages/setuptools/sandbox.py", line 254, in run_setup
        _execfile(setup_script, ns)
      File "/usr/local/lib/python3.5/site-packages/setuptools/sandbox.py", line 48, in _execfile
        exec(code, globals, locals)
      File "/tmp/easy_install-ocgjhe9m/numpy-1.13.0/setup.py", line 392, in <module>
        # higher up in this file.
      File "/tmp/easy_install-ocgjhe9m/numpy-1.13.0/setup.py", line 384, in setup_package
        if "--force" in sys.argv:
      File "/tmp/easy_install-ocgjhe9m/numpy-1.13.0/numpy/distutils/core.py", line 169, in setup
      File "/usr/local/lib/python3.5/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/local/lib/python3.5/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/local/lib/python3.5/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.5/site-packages/setuptools/command/bdist_egg.py", line 152, in run
        self.run_command("egg_info")
      File "/usr/local/lib/python3.5/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/local/lib/python3.5/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/easy_install-ocgjhe9m/numpy-1.13.0/numpy/distutils/command/egg_info.py", line 26, in run
      File "/usr/local/lib/python3.5/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/local/lib/python3.5/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/easy_install-ocgjhe9m/numpy-1.13.0/numpy/distutils/command/build_src.py", line 148, in run
      File "/tmp/easy_install-ocgjhe9m/numpy-1.13.0/numpy/distutils/command/build_src.py", line 159, in build_sources
      File "/tmp/easy_install-ocgjhe9m/numpy-1.13.0/numpy/distutils/command/build_src.py", line 294, in build_library_sources
      File "/tmp/easy_install-ocgjhe9m/numpy-1.13.0/numpy/distutils/command/build_src.py", line 377, in generate_sources
      File "numpy/core/setup.py", line 674, in get_mathlib_info
    RuntimeError: Broken toolchain: cannot link a simple C program

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-j8py_tat/scipy/setup.py", line 416, in <module>
        setup_package()
      File "/tmp/pip-build-j8py_tat/scipy/setup.py", line 412, in setup_package
        setup(**metadata)
      File "/usr/local/lib/python3.5/distutils/core.py", line 108, in setup
        _setup_distribution = dist = klass(attrs)
      File "/usr/local/lib/python3.5/site-packages/setuptools/dist.py", line 320, in __init__
        self.fetch_build_eggs(attrs['setup_requires'])
      File "/usr/local/lib/python3.5/site-packages/setuptools/dist.py", line 377, in fetch_build_eggs
        replace_conflicting=True,
      File "/usr/local/lib/python3.5/site-packages/pkg_resources/__init__.py", line 852, in resolve
        dist = best[req.key] = env.best_match(req, ws, installer)
      File "/usr/local/lib/python3.5/site-packages/pkg_resources/__init__.py", line 1124, in best_match
        return self.obtain(req, installer)
      File "/usr/local/lib/python3.5/site-packages/pkg_resources/__init__.py", line 1136, in obtain
        return installer(requirement)
      File "/usr/local/lib/python3.5/site-packages/setuptools/dist.py", line 445, in fetch_build_egg
        return cmd.easy_install(req)
      File "/usr/local/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 673, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/usr/local/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 699, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/usr/local/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 880, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/usr/local/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 1119, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/usr/local/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 1105, in run_setup
        run_setup(setup_script, args)
      File "/usr/local/lib/python3.5/site-packages/setuptools/sandbox.py", line 257, in run_setup
        raise
      File "/usr/local/lib/python3.5/contextlib.py", line 77, in __exit__
        self.gen.throw(type, value, traceback)
      File "/usr/local/lib/python3.5/site-packages/setuptools/sandbox.py", line 199, in setup_context
        yield
      File "/usr/local/lib/python3.5/contextlib.py", line 77, in __exit__
        self.gen.throw(type, value, traceback)
      File "/usr/local/lib/python3.5/site-packages/setuptools/sandbox.py", line 170, in save_modules
        saved_exc.resume()
      File "/usr/local/lib/python3.5/site-packages/setuptools/sandbox.py", line 145, in resume
        six.reraise(type, exc, self._tb)
      File "/usr/local/lib/python3.5/site-packages/pkg_resources/_vendor/six.py", line 685, in reraise
        raise value.with_traceback(tb)
      File "/usr/local/lib/python3.5/site-packages/setuptools/sandbox.py", line 158, in save_modules
        yield saved
      File "/usr/local/lib/python3.5/site-packages/setuptools/sandbox.py", line 199, in setup_context
        yield
      File "/usr/local/lib/python3.5/site-packages/setuptools/sandbox.py", line 254, in run_setup
        _execfile(setup_script, ns)
      File "/usr/local/lib/python3.5/site-packages/setuptools/sandbox.py", line 48, in _execfile
        exec(code, globals, locals)
      File "/tmp/easy_install-ocgjhe9m/numpy-1.13.0/setup.py", line 392, in <module>
        # higher up in this file.
      File "/tmp/easy_install-ocgjhe9m/numpy-1.13.0/setup.py", line 384, in setup_package
        if "--force" in sys.argv:
      File "/tmp/easy_install-ocgjhe9m/numpy-1.13.0/numpy/distutils/core.py", line 169, in setup
      File "/usr/local/lib/python3.5/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/local/lib/python3.5/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/local/lib/python3.5/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.5/site-packages/setuptools/command/bdist_egg.py", line 152, in run
        self.run_command("egg_info")
      File "/usr/local/lib/python3.5/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/local/lib/python3.5/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/easy_install-ocgjhe9m/numpy-1.13.0/numpy/distutils/command/egg_info.py", line 26, in run
      File "/usr/local/lib/python3.5/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/local/lib/python3.5/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/easy_install-ocgjhe9m/numpy-1.13.0/numpy/distutils/command/build_src.py", line 148, in run
      File "/tmp/easy_install-ocgjhe9m/numpy-1.13.0/numpy/distutils/command/build_src.py", line 159, in build_sources
      File "/tmp/easy_install-ocgjhe9m/numpy-1.13.0/numpy/distutils/command/build_src.py", line 294, in build_library_sources
      File "/tmp/easy_install-ocgjhe9m/numpy-1.13.0/numpy/distutils/command/build_src.py", line 377, in generate_sources
      File "numpy/core/setup.py", line 674, in get_mathlib_info
    RuntimeError: Broken toolchain: cannot link a simple C program
    /bin/sh: gcc: not found
    /bin/sh: gcc: not found

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-j8py_tat/scipy/
Removing intermediate container a86d07e229d7
The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1 </pre>

I'm using the python:3.5-alpine image. The versions of the packages are numpy==1.12.1, scipy==0.19.0, and gensim==2.1.0.

Answer 1

To install numpy, scipy, or gensim, the following lines need to be added to the Dockerfile before RUN pip install -r requirements.txt:

RUN apt-get -y install libc-dev
RUN apt-get -y install build-essential
RUN pip install -U pip

The first two lines install the C build tools required for the libraries. The tihrd line upgrades pip to the newest available version.