Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Differences between distribute, distutils, setuptools and distutils2?

The Situation

I’m trying to port an open-source library to Python 3. (SymPy, if anyone is wondering.)

So, I need to run 2to3 automatically when building for Python 3. To do that, I need to use distribute. Therefore, I need to port the current system, which (according to the doctest) is distutils.


The Problem

Unfortunately, I’m not sure what’s the difference between these modules—distutils, distribute, setuptools. The documentation is sketchy as best, as they all seem to be a fork of one another, intended to be compatible in most circumstances (but actually, not all)…and so on, and so forth.


The Question

Could someone explain the differences? What am I supposed to use? What is the most modern solution? (As an aside, I’d also appreciate some guide on porting to Distribute, but that’s a tad beyond the scope of the question…)

like image 591
VPeric Avatar asked Jun 14 '11 13:06

VPeric


People also ask

What is distutils?

The distutils package provides support for building and installing additional modules into a Python installation. The new modules may be either 100%-pure Python, or may be extension modules written in C, or may be collections of Python packages which include modules coded in both Python and C.

What is the use of Setuptools?

Setuptools is a package development process library designed to facilitate packaging Python projects by enhancing the Python standard library distutils (distribution utilities). It includes: Python package and module definitions. Distribution package metadata.

Is distutils deprecated?

In Python 3.10 and 3.11, distutils will be formally marked as deprecated. All known issues will be closed at this time. import distutils will raise a deprecation warning. New issues that would be considered release blocking may still be fixed, but support for new tools or platforms will not be added.

What is Setuptools package in Python?

Setuptools is a collection of enhancements to the Python distutils that allow developers to more easily build and distribute Python packages, especially ones that have dependencies on other packages. Packages built and distributed using setuptools look to the user like ordinary Python packages based on the distutils .


1 Answers

As of March 2020, most of the other answers to this question are several years out-of-date. When you come across advice on Python packaging issues, remember to look at the date of publication, and don't trust out-of-date information.

The Python Packaging User Guide is worth a read. Every page has a "last updated" date displayed, so you can check the recency of the manual, and it's quite comprehensive. The fact that it's hosted on a subdomain of python.org of the Python Software Foundation just adds credence to it. The Project Summaries page is especially relevant here.

Summary of tools:

Here's a summary of the Python packaging landscape:

Supported tools:

  • distutils is still the standard tool for packaging in Python. It is included in the standard library (Python 2 and Python 3). It is useful for simple Python distributions, but lacks features. It introduces the distutils Python package that can be imported in your setup.py script.

    • Official docs | distutils section of Python Package User Guide
  • setuptools was developed to overcome Distutils' limitations, and is not included in the standard library. It introduced a command-line utility called easy_install. It also introduced the setuptools Python package that can be imported in your setup.py script, and the pkg_resources Python package that can be imported in your code to locate data files installed with a distribution. One of its gotchas is that it monkey-patches the distutils Python package. It should work well with pip. It sees regular releases.

    • Official docs | Pypi page | GitHub repo | setuptools section of Python Package User Guide
  • scikit-build is an improved build system generator that internally uses CMake to build compiled Python extensions. Because scikit-build isn't based on distutils, it doesn't really have any of its limitations. When ninja-build is present, scikit-build can compile large projects over three times faster than the alternatives. It should work well with pip.

    • Official docs | Pypi page | GitHub repo | scikit-build section of Python Package User Guide
  • distlib is a library that provides functionality that is used by higher level tools like pip.

    • Official Docs | Pypi page | Bitbucket repo | distlib section of Python Package User Guide
  • packaging is also a library that provides functionality used by higher level tools like pip and setuptools

    • Official Docs | Pypi page | GitHub repo | packaging section of Python Package User Guide

Deprecated/abandoned tools:

  • distribute was a fork of setuptools. It shared the same namespace, so if you had Distribute installed, import setuptools would actually import the package distributed with Distribute. Distribute was merged back into Setuptools 0.7, so you don't need to use Distribute any more. In fact, the version on Pypi is just a compatibility layer that installs Setuptools.

  • distutils2 was an attempt to take the best of distutils, setuptools and distribute and become the standard tool included in Python's standard library. The idea was that distutils2 would be distributed for old Python versions, and that distutils2 would be renamed to packaging for Python 3.3, which would include it in its standard library. These plans did not go as intended, however, and currently, distutils2 is an abandoned project. The latest release was in March 2012, and its Pypi home page has finally been updated to reflect its death.

Others:

There are other tools, if you are interested, read Project Summaries in the Python Packaging User Guide. I won't list them all, to not repeat that page, and to keep the answer matching the question, which was only about distribute, distutils, setuptools and distutils2.

Recommendation:

If all of this is new to you, and you don't know where to start, I would recommend learning setuptools, along with pip and virtualenv, which all work very well together.

If you're looking into virtualenv, you might be interested in this question: What is the difference between venv, pyvenv, pyenv, virtualenv, virtualenvwrapper, etc?. (Yes, I know, I groan with you.)

like image 87
Flimm Avatar answered Sep 24 '22 15:09

Flimm