Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Finding a file in a Python module distribution [duplicate]

I've written a Python package that includes a bsddb database of pre-computed values for one of the more time-consuming computations. For simplicity, my setup script installs the database file in the same directory as the code which accesses the database (on Unix, something like /usr/lib/python2.5/site-packages/mypackage/).

How do I store the final location of the database file so my code can access it? Right now, I'm using a hack based on the __file__ variable in the module which accesses the database:

 dbname = os.path.join(os.path.dirname(__file__), "database.dat") 

It works, but it seems... hackish. Is there a better way to do this? I'd like to have the setup script just grab the final installation location from the distutils module and stuff it into a "dbconfig.py" file that gets installed alongside the code that accesses the database.

like image 750
Paul Avatar asked Sep 02 '08 09:09

Paul


People also ask

How do I read a static file inside a python package?

To access a file inside the current module, set the package argument to __package__ , e.g. pkg_resources. read_text(__package__, 'temp_file') (thanks to @ben-mares). Things become interesting when an actual filename is asked with path() , since now context-managers are used for temporarily-created files (read this).

What is Pkg_resources?

pkg_resources is a module used to find and manage Python package/version dependencies and access bundled files and resources, including those inside of zipped . egg files.

How do I find out what modules are in a python package?

First, it searches for the module in the current directory. If the module isn't found in the current directory, Python then searches each directory in the shell variable PYTHONPATH. The PYTHONPATH is an environment variable, consisting of a list of directories.

Where are Python data packages stored?

Place the files that you want to include in the package directory (in our case, the data has to reside in the roman/ directory). Add the field include_package_data=True in setup.py. Add the field package_data={'': [... patterns for files you want to include, relative to package dir...]} in setup.py .


2 Answers

Try using pkg_resources, which is part of setuptools (and available on all of the pythons I have access to right now):

>>> import pkg_resources >>> pkg_resources.resource_filename(__name__, "foo.config") 'foo.config' >>> pkg_resources.resource_filename('tempfile', "foo.config") '/usr/lib/python2.4/foo.config' 

There's more discussion about using pkg_resources to get resources on the eggs page and the pkg_resources page.

Also note, where possible it's probably advisable to use pkg_resources.resource_stream or pkg_resources.resource_string because if the package is part of an egg, resource_filename will copy the file to a temporary directory.

like image 141
Aaron Maenpaa Avatar answered Oct 06 '22 09:10

Aaron Maenpaa


Use pkgutil.get_data. It’s the cousin of pkg_resources.resource_stream, but in the standard library, and should work with flat filesystem installs as well as zipped packages and other importers.

like image 29
merwok Avatar answered Oct 06 '22 08:10

merwok