Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to build a source distribution without using setup.py file?

With the following package structure

.
├── my_package
│   └── __init__.py
├── setup.cfg
└── setup.py

Contents of setup.py

from setuptools import setup
setup()

Contents of setup.cfg

[metadata]
name = my_package
version = 0.1

[options]
packages = find:

I can build wheel or a source distribution for my_package like this

pip wheel --no-deps -w dist .
# generates file ./dist/my_package-0.1-py3-none-any.whl
python setup.py sdist
# generates file ./dist/my_package-0.1.tar.gz

But according to maintainer of setuptools, a declarative build configuration is ideal and using an imperative build is going to be a code smell. So we replace setup.py with pyproject.toml:

.
├── my_package
│   └── __init__.py
├── setup.cfg
└── pyproject.toml

Contents of pyproject.toml

[build-system]
build-backend = "setuptools.build_meta"
requires = ["setuptools", "wheel"]

And you can still build a wheel the same way as before, it works. But sdist doesn't work:

python: can't open file 'setup.py': [Errno 2] No such file or directory

So how should you actually build the .tar.gz file using setuptools? What's the user-facing tool to create sdist? I do not want to change the build backend. It looks like other packaging tools all write their own build entry points, but I thought the whole point of defining a declarative build system in the metadata was so that you didn't have to get hands-on with the build system, learning how each different packaging tool expects to be invoked or having to go into the interpreter and calling a Python API manually. But the PEP for build system requirements is over 2 years old now. Am I missing something obvious here?

How to build a source distribution without using setup.py file?

like image 803
platypus Avatar asked Nov 07 '19 17:11

platypus


People also ask

What is the command used for building a source distribution in python?

To build a source distribution, use the command line to navigate to the directory containing setup.py, and run the command python setup.py sdist.

Does pip need setup py?

As a first step, pip needs to get metadata about a package (name, version, dependencies, and more). It collects this by calling setup.py egg_info . The egg_info command generates the metadata for the package, which pip can then consume and proceed to gather all the dependencies of the package.

Why do we need setup py?

Use of Setup.py It primarily serves two purposes: It includes choices and metadata about the program, such as the package name, version, author, license, minimal dependencies, entry points, data files, and so on. Secondly, it serves as the command line interface via which packaging commands may be executed.


2 Answers

This is a somewhat controversial topic, and the answer for the moment is that there is no one single tool that everyone agrees is the "right way" to build source distributions, nor what that tool would be. You can see a long thread about it on the Python Packaging discourse.

I hesitate to give too much packaging advice in durable formats because the sands are always shifting, but as of November 2019, setup.py sdist is not deprecated, but it does have all the downsides that PEP 517 and PEP 518 were intended to fix - namely that you have to create the build environment yourself (and know about all the build dependencies), and it only works with setuptools/distutils and their equivalents.

It is not an "official" recommendation, but the current (Dec. 2020) best replacement for setup.py sdist and setup.py bdist_wheel is using pypa-build. Install once with

pip install build

and use as

python -m build --sdist --wheel

This builds source distribution and wheel at the same time. This is how I build my PEP 517-compatible packages.

This requires that your project have a pyproject.toml, and the pyproject.toml must have build-system.requires and build-system.build-backend keys, but it will work for any project with a PEP 517-compatible backend (including flit).

Other tools

Why not use flit or poetry or hatch? Those tools are all available for those who want to use them, but they are not an answer to this question. This question is asking about projects build with setuptools that use the declarative setup.cfg format. Neither flit nor poetry act as generic PEP 517 build front-ends, and so they only work as build commands for projects using their respective backends.

I am not familiar enough with hatch to say whether or not it can manage projects with backends other than setuptools, but (again, as of November 2019), it is not a PEP 517 frontend, and it will not work if you don't have a setup.py (it will raise the error "can't open file setup.py", and it will ignore your pyproject.toml file).

like image 165
Paul Avatar answered Oct 14 '22 05:10

Paul


There's nothing "obvious" when it comes to Python packaging. Indeed, for the time being, at least if you are using distutils/setuptools it is necessary to create a (nearly) empty setup.py file, even if you are using a fully declarative setup.cfg:

#!/usr/bin/env python
from setuptools import setup
setup()

I also recommend chmod +x setup.py.

In this case you are just writing the "entry point" to the build system yourself, and setup() is just the main() function for it--but now all the arguments that were traditionally passed to setup() can be read from setup.cfg instead.

Now you can still use setup.py sdist if you want to make a source tarball:

./setup.py sdist

You could also try one of the alternative build systems that are enabled via pyproject.toml, such as Flit.

like image 1
Iguananaut Avatar answered Oct 14 '22 06:10

Iguananaut