Reliable way to get the "build" directory from within setup.py

Question

Inside the setup.py script I need to create some temporary files for the installation. The natural place to put them would be the "build/" directory.

Is there a way to retrieve its path that works if installing via pypi, from source, easy_install, pip, ...?

Thanks a lot!

Answer 1

By default distutils create build/ in current working dir, but it can be changed by argument --build-base. Seems like distutils parses it when executing setup and parsed argument does not accessible from outside, but you can cut it yourself:

import sys
build_base_long = [arg[12:].strip("= ") for arg in sys.argv if arg.startswith("--build-base")]
build_base_short = [arg[2:].strip(" ") for arg in sys.argv if arg.startswith("-b")]
build_base_arg = build_base_long or build_base_short
if build_base_arg:
    build_base = build_base_arg[0]
else:
    build_base = "."

This naive version of parser still shorter than optparse's version with proper error handling for unknown flags. Also you can use argparse's parser, which have try_parse method.

Answer 2

distutils/setuptools provide an abstract Command class that users can use to add custom commands to their package's setup process. This is the same class that built-in setup commands like build and install are subclasses of.

Every class that is a subclass of the abstract Command class must implement the initialize_options, finalize_options, and run methods. The "options" these method names refer to are class attributes that are derived from command-line arguments provided by the user (they can also have default values). The initialize_options method is where a class's options are defined, the finalize_options method is where a class's option values are assigned, and the run method is where a class's option values are used to perform the function of the command.

Since command-line arguments may affect more than one command, some command classes may share options with other command classes. For example, all the distutils/setuptools build commands (build, build_py, build_clib, build_ext, and build_scripts) and the install command need to know where the build directory is. Instead of having every one of these command classes define and parse the same command-line arguments into the same options, the build command, which is the first of all these commands to be executed, defines and parses the command-line arguments and options, and all the other classes get the option values from the build command in their finalize_options method.

For example, the build class defines the build_base and build_lib options in its initialize_options method and then computes their values from the command-line arguments in its finalize_options method. The install classes also defines the build_base and build_lib options in its initialize_options method but it gets the values for these options from the build command in its finalize_options method.

You can use the same pattern to add a custom sub-command to the build command as follows (it would be similar for install)

import setuptools
from distutils.command.build import build


class BuildSomething(setuptools.Command):

  def initialize_options(self):
    # define the command's options
    self.build_base = None
    self.build_lib = None
  
  def finalize_options(self):
    # get the option values from the build command
    self.set_undefined_options('build',
                               ('build_base', 'build_base'),
                               ('build_lib', 'build_lib'))

  def run(self):
    # do something with the option values
    print(self.build_base)  # defaults to 'build'
    print(self.build_lib)


build_something_command = 'build_something'


class Build(build):

  def has_something(self):
    # update this to check if your build should run
    return True

  sub_commands =  [(build_something_command, has_something)] + build.sub_commands


COMMAND_CLASS = {
  build_something_command: BuildSomething,  # custom command
  'build': Build  # override distutils/setuptools build command
}


setuptools.setup(cmdclass=COMMAND_CLASS)

Alternatively, you could just subclass one of the distutils/setuptools classes if you just want to extend its functionality and it already has the options you need

import setuptools
from setuptools.command.build_py import build_py


class BuildPy(build_py):

  def initialize_options(self):
    pass
  
  def finalize_options(self):
    pass

  def run(self):
    # do something with the option values
    print(self.build_lib)  # inherited from build_py
    build_py.run(self)  # make sure the regular build_py still runs


COMMAND_CLASS = {
  'build_py': BuildPy  # override distutils/setuptools build_py command
}


setuptools.setup(cmdclass=COMMAND_CLASS)

Unfortunately, none of this is very well documented anywhere. I learned most of it from reading the distutils and setuptools source code. Any of the build*.py and install*.py files in either repository's command directory is informative. The abstract Command class is defined in distutils.