How to build a source distribution without using setup.py file?

Question

With the following package structure

.
├── my_package
│   └── __init__.py
├── setup.cfg
└── setup.py

Contents of setup.py

from setuptools import setup
setup()

Contents of setup.cfg

[metadata]
name = my_package
version = 0.1

[options]
packages = find:

I can build wheel or a source distribution for my_package like this

pip wheel --no-deps -w dist .
# generates file ./dist/my_package-0.1-py3-none-any.whl
python setup.py sdist
# generates file ./dist/my_package-0.1.tar.gz

But according to maintainer of setuptools, a declarative build configuration is ideal and using an imperative build is going to be a code smell. So we replace setup.py with pyproject.toml:

.
├── my_package
│   └── __init__.py
├── setup.cfg
└── pyproject.toml

Contents of pyproject.toml

[build-system]
build-backend = "setuptools.build_meta"
requires = ["setuptools", "wheel"]

And you can still build a wheel the same way as before, it works. But sdist doesn't work:

python: can't open file 'setup.py': [Errno 2] No such file or directory

So how should you actually build the .tar.gz file using setuptools? What's the user-facing tool to create sdist? I do not want to change the build backend. It looks like other packaging tools all write their own build entry points, but I thought the whole point of defining a declarative build system in the metadata was so that you didn't have to get hands-on with the build system, learning how each different packaging tool expects to be invoked or having to go into the interpreter and calling a Python API manually. But the PEP for build system requirements is over 2 years old now. Am I missing something obvious here?

How to build a source distribution without using setup.py file?

Answer 1

This is a somewhat controversial topic, and the answer for the moment is that there is no one single tool that everyone agrees is the "right way" to build source distributions, nor what that tool would be. You can see a long thread about it on the Python Packaging discourse.

I hesitate to give too much packaging advice in durable formats because the sands are always shifting, but as of November 2019, setup.py sdist is not deprecated, but it does have all the downsides that PEP 517 and PEP 518 were intended to fix - namely that you have to create the build environment yourself (and know about all the build dependencies), and it only works with setuptools/distutils and their equivalents.

It is not an "official" recommendation, but the current (Dec. 2020) best replacement for setup.py sdist and setup.py bdist_wheel is using pypa-build. Install once with

pip install build

and use as

python -m build --sdist --wheel

This builds source distribution and wheel at the same time. This is how I build my PEP 517-compatible packages.

This requires that your project have a pyproject.toml, and the pyproject.toml must have build-system.requires and build-system.build-backend keys, but it will work for any project with a PEP 517-compatible backend (including flit).

Other tools

Why not use flit or poetry or hatch? Those tools are all available for those who want to use them, but they are not an answer to this question. This question is asking about projects build with setuptools that use the declarative setup.cfg format. Neither flit nor poetry act as generic PEP 517 build front-ends, and so they only work as build commands for projects using their respective backends.

I am not familiar enough with hatch to say whether or not it can manage projects with backends other than setuptools, but (again, as of November 2019), it is not a PEP 517 frontend, and it will not work if you don't have a setup.py (it will raise the error "can't open file setup.py", and it will ignore your pyproject.toml file).

Answer 2

There's nothing "obvious" when it comes to Python packaging. Indeed, for the time being, at least if you are using distutils/setuptools it is necessary to create a (nearly) empty setup.py file, even if you are using a fully declarative setup.cfg:

#!/usr/bin/env python
from setuptools import setup
setup()

I also recommend chmod +x setup.py.

In this case you are just writing the "entry point" to the build system yourself, and setup() is just the main() function for it--but now all the arguments that were traditionally passed to setup() can be read from setup.cfg instead.

Now you can still use setup.py sdist if you want to make a source tarball:

./setup.py sdist

You could also try one of the alternative build systems that are enabled via pyproject.toml, such as Flit.