Python setup.py to run shell script

Question

I need to run my own script during 'sdist' phase while creating Python package. I wrote the following script. Do you know better approach? Could you recommend please the better one or link to the official documentation on setuptools where this moment has been explained?

import subprocess
import sys

from setuptools import setup, find_packages, os

if 'sdist' in sys.argv:
    cwd = os.getcwd()
    os.chdir('website/static/stylesheets/')
    result = subprocess.call("scss --update --compass ./*.scss", shell=True)
    if result != 0:
        exit(1)
    os.chdir(cwd)

setup(name = "site",
    author="Vladimir Ignatev",
    author_email="[email protected]",
    version="0.1",
    packages=find_packages(),
    include_package_data=True,
    zip_safe=True,
)

Answer 1

Although this comes pretty late, here is a solution proposal.

Basically, it is simply subclassing the distutils' sdist command with adding custom logic and registering it in the setup function. Unfortunately, the official documentation to this topic is kind of vague and laconic; Extending Distutils provides at least a tiny example for the start. I found it much better to read the code of modules in distutils.command package to see how the actual commands are implemented.

To execute an arbitrary command, you can use the method distutils.cmd.Command::spawn that executes passed input string, raising a DistutilsExecError if the command's exit code in not zero:

from distutils.command.sdist import sdist as sdist_orig
from distutils.errors import DistutilsExecError

from setuptools import setup  


class sdist(sdist_orig):

    def run(self):
        try:
            self.spawn(['ls', '-l'])
        except DistutilsExecError:
            self.warn('listing directory failed')
        super().run()


setup(name='spam',
    version='0.1',
    packages=[],
    cmdclass={
        'sdist': sdist
    }
)

Running the setup script above yields:

$ python setup.py sdist
running sdist
ls -l
total 24
-rw-r--r--  1 hoefling  staff   52 23 Dez 19:06 MANIFEST
drwxr-xr-x  3 hoefling  staff   96 23 Dez 19:06 dist
-rw-r--r--  1 hoefling  staff  484 23 Dez 19:07 setup.py
running check
...
writing manifest file 'MANIFEST'
creating spam-0.1
making hard links in spam-0.1...
hard linking setup.py -> spam-0.1
Creating tar archive
removing 'spam-0.1' (and everything under it)

Reusing commands

Here is (although simplified) a real life example of a command we are using in our projects that is used around NodeJS projects and invokes yarn:

import distutils
import os
import pathlib
import setuptools

_YARN_CMD_SEP = ';'

_HELP_MSG_SUBCMD = (
    'yarn subcommands to execute (separated '
    'by {})'.format(_YARN_CMD_SEP)
)

_HELP_MSG_PREFIX = (
    'path to directory containing package.json. '
    'If not set, current directory is assumed.'
)


class yarn(setuptools.Command):

    description = ('runs yarn commands. Assumes yarn is '
                   'already installed by the user.')

    user_options = [
        ('subcommands=', None, _HELP_MSG_SUBCMD),
        ('prefix=', None, _HELP_MSG_PREFIX),
    ]

    def initialize_options(self) -> None:
        self.subcommands = []
        self.prefix = None  # type: pathlib.Path

    def finalize_options(self) -> None:
        self.subcommands = [
            cmd.strip() for cmd in str(self.subcommands).split(self._YARN_CMD_SEP)
        ]
        self.prefix = pathlib.Path(self.prefix) if self.prefix else pathlib.Path()

    def run(self) -> None:
        cwd = pathlib.Path().absolute()
        os.chdir(str(self.prefix.absolute()))  # change to prefix dir
        for cmd in self.subcommands:
            self.announce('running yarn {} ...'.format(cmd), level=distutils.log.INFO)
            self.spawn(['yarn'] + cmd.split(' '))
        os.chdir(str(cwd))  # change back to our previous dir

Example usage:

$ python setup.py yarn --prefix=. --subcommands="add leftpad; remove leftpad"
running yarn
running yarn add leftpad ...
yarn add leftpad
yarn add v1.3.2
warning package.json: No license field
warning No license field
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
success Saved lockfile.
success Saved 1 new dependency.
└─ [email protected]
warning No license field
✨  Done in 0.33s.
running yarn remove leftpad ...
yarn remove leftpad
yarn remove v1.3.2
warning package.json: No license field
[1/2] Removing module leftpad...
[2/2] Regenerating lockfile and installing missing dependencies...
warning No license field
success Uninstalled packages.
✨  Done in 0.13s.

You can also use the yarn in your command chain as every other command: python setup.py yarn test sdist etc.