How to get installation directory using setuptools and pkg_ressources

Question

I'm working on a package P with setuptools and pkg_resources, where the package, after installation, needs to download some binaries and place them in a dedicated directory (P/bin/).

I'm trying to use pkg_ressources.resource_filename to get the absolute directory path. (in order to work with virtualenv)

During the installation using python setup.py install, the pkg_ressources.resource_filename doesn't return a path like /home/user/tests/venv/lib/python3.4/site-package/P/bin/, but the path to the actual module, like /home/user/projects/P/P/bin/.

That's a problem, because i need the installation directory (inside the virtualenv), not my personal project directory (where i develop the module).

If i try to pass through pypi, using pip install module, the directory returned by pkg_ressources.resource_filename is a temporary file like /tmp/pip-build-xxxxxxx/P/bin/, which is again not the place where the binaries should be put.

Here is my setup.py:

from setuptools import setup
import os

from setuptools.command.install import install as _install
from pkg_resources import resource_filename


def post_install():
    """Get the binaries online, and give them the execution permission"""
    package_dir_bin = resource_filename('P', 'bin') # should be /home/user/tests/venv/lib/python3.4/site-package/P/bin/
    # package_dir_bin = resource_filename(Requirement.parse('P'), 'bin') # leads to same results
    put_binaries_in(package_dir_bin)
    os.system('chmod +x ' + package_dir_bin + '*')



class install(_install):
    # see http://stackoverflow.com/a/18159969

    def run(self):
        """Call superclass run method, then downloads the binaries"""
        _install.run(self)
        self.execute(post_install, args=[], msg=post_install.__doc__)


setup(
    cmdclass={'install': install},
    name = 'P',
    # many metadata
    package_dir = { 'P' : 'P'},
    package_data = {
        'P' : ['bin/*.txt'] # there is an empty txt file in bin directory
    },
)

Is there a standard way to get the installation directory during the installation, cross-platform and compatible python 2 and 3 ? If not, what should i do ?

Answer 1

A workaround is to use the site package instead of pkg_resources that seems not designed for access resources during installation.

Here is a function that detect the installation directory during installation:

import os, sys, site

def binaries_directory():
    """Return the installation directory, or None"""
    if '--user' in sys.argv:
        paths = (site.getusersitepackages(),)
    else:
        py_version = '%s.%s' % (sys.version_info[0], sys.version_info[1])
        paths = (s % (py_version) for s in (
            sys.prefix + '/lib/python%s/dist-packages/',
            sys.prefix + '/lib/python%s/site-packages/',
            sys.prefix + '/local/lib/python%s/dist-packages/',
            sys.prefix + '/local/lib/python%s/site-packages/',
            '/Library/Python/%s/site-packages/',
        ))

    for path in paths:
        if os.path.exists(path):
            return path
    print('no installation path found', file=sys.stderr)
    return None

This solution is not Python 2.7 compatible, in case of installation using virtualenv, because of the known bug about the module site. (see related SO)

Answer 2

The easiest solution is to follow the first snippet of this answer.

Essentially you just have to save the setup call in a variable and then look at its attributes. There are a few convenient ones

from setuptools import setup

s = setup(
    # ...
)

print(s.command_obj['install'].__dir__())

# [..., 'install_base', 'install_lib', 'install_script', ...]

Those I showed are, respectively, the equivalents of /usr, /usr/lib/pythonX.Y/site-packages and /usr/bin. But there are also other attributes that might be useful. Those are all absolute paths.