How to use Cython with Poetry?

Question

I have a file in my project which I would like to compile for performance reasons:

mylibrary/myfile.py

How to achieve this with Poetry?

Answer 1

There is an undocumented feature in Poetry. Add this to your pyproject.toml:

[tool.poetry]
...
build = 'build.py'

[build-system]
requires = ["poetry>=0.12", "cython"]
build-backend = "poetry.masonry.api"

What this does is runs the build.py:build() function inside the implicitly generated setup.py. This is where we build.

So, create a build.py that provides the build() function:

import os

# See if Cython is installed
try:
    from Cython.Build import cythonize
# Do nothing if Cython is not available
except ImportError:
    # Got to provide this function. Otherwise, poetry will fail
    def build(setup_kwargs):
        pass
# Cython is installed. Compile
else:
    from setuptools import Extension
    from setuptools.dist import Distribution
    from distutils.command.build_ext import build_ext

    # This function will be executed in setup.py:
    def build(setup_kwargs):
        # The file you want to compile
        extensions = [
            "mylibrary/myfile.py"
        ]

        # gcc arguments hack: enable optimizations
        os.environ['CFLAGS'] = '-O3'

        # Build
        setup_kwargs.update({
            'ext_modules': cythonize(
                extensions,
                language_level=3,
                compiler_directives={'linetrace': True},
            ),
            'cmdclass': {'build_ext': build_ext}
        })

Now, when you do poetry build, nothing happens. But if you install this package elsewhere, it gets compiled.

You can also build it manually with:

$ cythonize -X language_level=3 -a -i mylibrary/myfile.py

Finally, it seems that you can't publish binary packages to PyPi. The solution is to limit your build to "sdist":

$ poetry build -f sdist
$ poetry publish