pyproject.toml setuptools: How can I specify optional scripts (and modules)?

Question

I have a python project with some modules, scripts and optional dependencies:

[project.optional-dependencies]
extra = [ "tqdm", "antspyx>=0.4.2", "scikit-image", "mpi4py", "scipy" ]

[project.scripts]
reg = "myproject._cmd.reg:main"
conv = "myproject._cmd.conv:main"

[tool.setuptools.packages.find]
where = ["src"]
include = ["myproject*"]

Now I would like the scripts to be installed, only for pip install myproject[extra]. By now, these scripts are installed even for pip install myproject

It would also be nice to be able to split my modules like this. Modules in src/myproject/extra/ should be installed only for pip install myproject[extra]. If pip install myproject is called, only modules in src/myproject/modules/ should be installed.

So, how can I declare optional modules and optional scripts in pyproject.toml?

Answer 1

(DON'T USE THIS, IT WON'T WORK): Theoretically, this is the syntax:

[project.scripts]
# This is always available.
run = "test_project.launcher:run"
# (DON'T USE THIS, IT WON'T WORK):
run_extra = "test_project.launcher:run_gui [extra]"

HOWEVER, from the PyPA user guide, this is deprecated:

There is also an optional property: the extras are a set of strings identifying optional features of the distribution providing the entry point. If these are specified, the entry point requires the dependencies of those ‘extras’. See the metadata field Provides-Extra (multiple use).

Using extras for an entry point is no longer recommended. Consumers should support parsing them from existing distributions, but may then ignore them. New publishing tools need not support specifying extras. The functionality of handling extras was tied to setuptools’ model of managing ‘egg’ packages, but newer tools such as pip and virtualenv use a different model.

And in practice, all buildsystems I tried would always make a run_extra script available regardless of the dependencies.

This thread on the python message board is the most extensive discussion I could find, and it seems like the primary recommendations are:

Have your entrypoint print install instructions:

def run_extra():
    try:
        import optional_dependency
    except ImportError:
        print(f"Please install 'myproject[extra]' to use this command.")
        sys.exit(0)

    _run_extra()

Split it into two packages

If you have some large amount of code, entrypoints, and dependencies that only need to exist if an option is installed, maybe what you actually want is to have a "core" or "lib" package and a "cli" or "gui" package that depends on lib. With editable installs the imports will work while you're developing.

I believe this is buildsystem specific. UV has a workspace concept. Or you could have a requirements.txt file:

-e ./core
-e ./extras