setup_requires with Cython?

Question

I'm creating a setup.py file for a project with some Cython extension modules.

I've already gotten this to work:

from setuptools import setup, Extension
from Cython.Build import cythonize

setup(
    name=...,
    ...,
    ext_modules=cythonize([ ... ]),
)

This installs fine. However, this assumes Cython is installed. What if it's not installed? I understand this is what the setup_requires parameter is for:

from setuptools import setup, Extension
from Cython.Build import cythonize

setup(
    name=...,
    ...,
    setup_requires=['Cython'],
    ...,
    ext_modules=cythonize([ ... ]),
)

However, if Cython isn't already installed, this will of course fail:

$ python setup.py install
Traceback (most recent call last):
  File "setup.py", line 2, in <module>
    from Cython.Build import cythonize
ImportError: No module named Cython.Build

What's the proper way to do this? I need to somehow import Cython only after the setup_requires step runs, but I need Cython in order to specify the ext_modules values.

Answer 1

Starting from 18.0 release of setuptools (released on 2015-06-23) it is possible to specify Cython in setup_requires and pass *.pyx modules sources for regular setuptools.Extension:

from setuptools import setup, Extension


setup(
    # ...
    setup_requires=[
        # Setuptools 18.0 properly handles Cython extensions.
        'setuptools>=18.0',
        'cython',
    ],
    ext_modules=[
        Extension(
            'mylib',
            sources=['src/mylib.pyx'],
        ),
    ],
)

Answer 2

You must wrap the from Cython.Build import cythonize in a try-except, and in the except, define cythonize as a dummy function. This way the script can be loaded without failing with an ImportError.

Then later when the setup_requires argument is handled, Cython will be installed and the setup script will be re-executed. Since at that point Cython is installed, you'll be able to successfully import cythonize

try:
    from Cython.Build import cythonize
except ImportError:
     def cythonize(*args, **kwargs):
         from Cython.Build import cythonize
         return cythonize(*args, **kwargs)

EDIT

As noted in comments, after setuptools deals with missing dependencies, it won't re-load Cython. I hadn't thought of it before, but you could also try a late-binding approach to stubbing out cythonize

Answer 3

There seems to be a third way of having build dependencies installed before executing the actual setup.py described here (requires pip):

https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html#basic-setup-

In essence:

1. Create the file pyproject.toml with the following contents:

[build-system]
requires = ["setuptools", "wheel", "Cython"]

2. Use pip install -e . for setting things up