Cython "ModuleNotFoundError" even though it was just installed via requirements.txt while building docker image

Question

Installing packages when building a docker image via the requirements.txt. In the Dockerfile it reads

COPY requirements.txt /app/requirements.txt
RUN pip3 install -r /app/requirements.txt

In the requirements.txt I put down deploy token for installing a private git repo, such as this example one:

git+https://gitlab+deploy-token-123456:[email protected]/example/lib.git@a1b2c3d4e5f6g7h8j9k1l2m3n4o5p6q7r8s9t

It throws an error, ModuleNotFoundError: No module named 'Cython'.

However, even though I include Cython in the requirements and it gets collected correctly, I get the same error.

Here is the complete message:

docker build -t docker.example/example:dev .
Sending build context to Docker daemon  8.704kB
Step 1/7 : FROM python:3.6.6-alpine
 ---> a78e257617d1
Step 2/7 : RUN apk update
 ---> Using cache
 ---> 35e6b7308e61
Step 3/7 : RUN apk add gcc build-base musl-dev libc-dev util-linux-dev linux-headers python3-dev git ca-certificates libffi-dev bash postgresql-libs postgresql-dev
 ---> Using cache
 ---> 5ca54839bb7c
Step 4/7 : RUN python3 -m ensurepip
 ---> Using cache
 ---> 03d95f700108
Step 5/7 : RUN pip install --upgrade pip
 ---> Using cache
 ---> e462d855f07f
Step 6/7 : COPY requirements.txt /app/requirements.txt
 ---> f69aa355732a
Step 7/7 : RUN pip3 install -r /app/requirements.txt
 ---> Running in d75911af46c7
Collecting git+https://gitlab%2Bdeploy-token-171703:****@gitlab.com/example/lib.git@f75285be6ee56360d6f0cfbf64f09a86819def5c (from -r /app/requirements.txt (line 2))
  Cloning https://gitlab%2Bdeploy-token-171703:****@gitlab.com/example/lib.git (to revision f75285be6ee56360d6f0cfbf64f09a86819def5c) to /tmp/pip-req-build-yt5mk7q5
  Running command git clone -q 'https://gitlab%2Bdeploy-token-171703:****@gitlab.com/example/lib.git' /tmp/pip-req-build-yt5mk7q5
Collecting Cython==0.29.17
  Downloading Cython-0.29.17-py2.py3-none-any.whl (971 kB)
Collecting psychrolib==2.5.0
  Downloading PsychroLib-2.5.0.zip (9.5 kB)
Collecting influxdb==5.2.2
  Downloading influxdb-5.2.2-py2.py3-none-any.whl (70 kB)
Collecting configparser==3.5.0
  Downloading configparser-3.5.0.tar.gz (39 kB)
Collecting reportlab
  Downloading reportlab-3.5.42.tar.gz (2.9 MB)
Collecting PyPDF2
  Downloading PyPDF2-1.26.0.tar.gz (77 kB)
Collecting matplotlib>=2.2.2
  Downloading matplotlib-3.2.1.tar.gz (40.3 MB)


    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-mqz7vwwc/matplotlib/setup.py'"'"'; __file__='"'"'/tmp/pip-install-mqz7vwwc/matplotlib/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-mqz7vwwc/matplotlib/pip-egg-info
         cwd: /tmp/pip-install-mqz7vwwc/matplotlib/
    Complete output (112 lines):
    Processing numpy/random/_bounded_integers.pxd.in
    Processing numpy/random/_bounded_integers.pyx.in
    Traceback (most recent call last):
      File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 61, in process_pyx
        from Cython.Compiler.Version import version as cython_version
    ModuleNotFoundError: No module named 'Cython'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 238, in <module>
        main()
      File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 234, in main
        find_process_files(root_dir)
      File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 225, in find_process_files
        process(root_dir, fromfile, tofile, function, hash_db)
      File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 191, in process
        processor_function(fromfile, tofile)
      File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 94, in process_tempita_pyx
        process_pyx(pyxfile, tofile)
      File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/tools/cythonize.py", line 66, in process_pyx
        raise OSError('Cython needs to be installed in Python as a module')
    OSError: Cython needs to be installed in Python as a module
    Running from numpy source directory.
    /tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py:461: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
      run_build = parse_setuppy_commands()
    Traceback (most recent call last):
      File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 154, in save_modules
        yield saved
      File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 250, in run_setup
        _execfile(setup_script, ns)
      File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 45, in _execfile
        exec(code, globals, locals)
      File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 488, in <module>
      File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 469, in setup_package
      File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 275, in generate_cython
        )
    RuntimeError: Running cythonize failed!

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-mqz7vwwc/matplotlib/setup.py", line 274, in <module>
        cmdclass=cmdclass,
      File "/usr/local/lib/python3.6/site-packages/setuptools/__init__.py", line 139, in setup
        _install_setup_requires(attrs)
      File "/usr/local/lib/python3.6/site-packages/setuptools/__init__.py", line 134, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/usr/local/lib/python3.6/site-packages/setuptools/dist.py", line 514, in fetch_build_eggs
        replace_conflicting=True,
      File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 777, in resolve
        replace_conflicting=replace_conflicting
      File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1060, in best_match
        return self.obtain(req, installer)
      File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1072, in obtain
        return installer(requirement)
      File "/usr/local/lib/python3.6/site-packages/setuptools/dist.py", line 581, in fetch_build_egg
        return cmd.easy_install(req)
      File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 676, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 702, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 887, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 1155, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 1141, in run_setup
        run_setup(setup_script, args)
      File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 253, in run_setup
        raise
      File "/usr/local/lib/python3.6/contextlib.py", line 99, in __exit__
        self.gen.throw(type, value, traceback)
      File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "/usr/local/lib/python3.6/contextlib.py", line 99, in __exit__
        self.gen.throw(type, value, traceback)
      File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 166, in save_modules
        saved_exc.resume()
      File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 141, in resume
        six.reraise(type, exc, self._tb)
      File "/usr/local/lib/python3.6/site-packages/setuptools/_vendor/six.py", line 685, in reraise
        raise value.with_traceback(tb)
      File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 154, in save_modules
        yield saved
      File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 250, in run_setup
        _execfile(setup_script, ns)
      File "/usr/local/lib/python3.6/site-packages/setuptools/sandbox.py", line 45, in _execfile
        exec(code, globals, locals)
      File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 488, in <module>
      File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 469, in setup_package
      File "/tmp/easy_install-ho_rizu1/numpy-1.18.4/setup.py", line 275, in generate_cython
        )
    RuntimeError: Running cythonize failed!

    Edit setup.cfg to change the build options; suppress output with --quiet.

    BUILDING MATPLOTLIB
      matplotlib: yes [3.2.1]
          python: yes [3.6.6 (default, Sep 12 2018, 02:15:29)  [GCC 6.4.0]]
        platform: yes [linux]
     sample_data: yes [installing]
           tests: no  [skipping due to configuration]
             agg: yes [installing]
           tkagg: yes [installing; run-time loading from Python Tcl/Tk]
          macosx: no  [Mac OS-X only]

    Cythonizing sources
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
WARNING: You are using pip version 20.0.2; however, version 20.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
The command '/bin/sh -c pip3 install -r /app/requirements.txt' returned a non-zero code: 1
Makefile:8: recipe for target 'build' failed
make: *** [build] Error 1

Answer 1

I was able to resolve the problem by including

RUN pip3 install Cython

in the Dockerfile, above the requirements installation.