Installing data_files in setup.py with pip install -e

Question

I'm trying to provide a bash completion script for my CLI tool that is written in Python. According to the Python Packaging Authority, data_files in setup.py is exactly what I need:

Although configuring package_data is sufficient for most needs, in some cases you may need to place data files outside of your packages. The data_files directive allows you to do that. It is mostly useful if you need to install files which are used by other programs, which may be unaware of Python packages.

So I added the completion file like this:

data_files=[
    ('/usr/share/bash-completion/completions', ['completion/dotenv']),
],

and try to test it with:

pip install -e .

In my virtual environment. However, the completion script gets not installed. Did I forgot something or is pip broken? The full project can be found here

Answer 1

I had the same issue and I have implemented a workaround.

It seems to me that python setup.py develop or (pip install -e .) does not run the same function than python setup.py install. In fact, I have noticed by looking in the source code that python setup.py install run build_py :

https://github.com/python/cpython/blob/master/Lib/distutils/command/build_py.py#L134 https://github.com/pypa/setuptools/blob/master/setuptools/command/build_py.py

After a few digging I have opted to override the develop command as follow. The following code is python3.6:

""" SetupTool Entry Point """
import sys
from pathlib import Path
from shutil import copy2

from setuptools import find_packages, setup
from setuptools.command.develop import develop

# create os_data_files that will be used by the default install command
os_data_files = [
    (
        f"{sys.prefix}/config",  # providing absolute path, sys.prefix will be different in venv
        [
            "src/my_package/config/properties.env",
        ],
    ),        
]


def build_package_data():
    """ implement the necessary function for develop """
    for dest_dir, filenames in os_data_files:
        for filename in filenames:
            print(
                "CUSTOM SETUP.PY (build_package_data): copy %s to %s"
                % (filename, dest_dir)
            )
            copy2(filename, dest_dir)


def make_dirstruct():
    """ Set the the logging path """
    for subdir in ["config"]:
        print("CUSTOM SETUP.PY (make_dirstruct): creating %s" % subdir)
        (Path(BASE_DIR) / subdir).mkdir(parents=True, exist_ok=True)


class CustomDevelopCommand(develop):
    """ Customized setuptools install command """

    def run(self):
        develop.run(self)
        make_dirstruct()
        build_package_data()

# provide the relevant information for stackoverflow
setup(        
    package_dir={"": "src"},
    packages=find_packages("src"),
    data_files=os_data_files,                
    cmdclass={"develop": CustomDevelopCommand},
)