Accessing files in python egg from inside the egg

Question

The question is an attempt to get the exact instruction on how to do that. There were few attempts before, which don't seem to be full solutions:

solution to move the file inside the package

solution to read as zip

accessing meta info via get_distribution

The task at hand is to read the information about the egg the program is running from. There are few ways as i understand:

1. hard code the location of the egg and treat it as zip archive - will work, but not flexible enough, because it will need to be edited and recompiled in case if file is moved to another location

2. use ResourceManager().resource_filename(__name__, filename) - this seems to be limited in the fact that i cannot access the file that is inside the egg, but not inside the package. notations like "../../EGG-INFO/PKG-INFO" in filename don't work giving KeyError. So no good either.

3. use dist = pkg_resources.get_distribution("dist_name") and then use dist object to get information, but I cannot understand from the docs how should i specify my distribution name? It can't find it.

So, i'm looking for correct solution about using pkg_resources.get_distribution plus it would be nice to finally have a full solution to read any file from inside the egg.

Thanks!

Answer 1

Setuptools/distribute/pkg_resources is designed to be a sort of transparent overlay to standard Python distutils, which are pretty limited and don't allow a good way of distributing code.

eggs are just a way of putting together a bunch of python files, data files, and metadata, somewhat similar to Java JARs - but python packages can be installed from source even without en egg (which is a concept which does not exist in the standard distribution).

So there two scenarios here: either you're a programmer which is trying to use some file inside a library, and in such case, in order to read any file from your distribution, you don't need its full path - you just need an open file object with its content, right? So you should do something like this:

from pkg_resources import resource_stream, Requirement
resource_stream(Requirement.parse("restez==0.3.2"), "restez/httpconn.py")

That will return an open, readable file of the file you requested from your package distribution. If it's a zipped egg, it will be automatically be extracted.

Please note that you should specify the package name inside (restez) because the distribution name may be different from the package (e.g. distribution Twisted then uses twisted package name). Requirements parsing use this syntax: http://setuptools.readthedocs.io/en/latest/pkg_resources.html#requirements-parsing

This should suffice - you shouldn't need to know the path of the egg once you know how to fetch files from inside the egg.

If you really want the full path and you're sure your egg is uncompressed, use resource_filename instead of resource_stream.

Otherwise, if you're building a "packaging tool" and need to access the contents of your package, be it an egg or anything, you'll have to do that yourself by hand, just like pkg_resources does (pkg_resources source) . There's not a precise API for "querying an egg content" because there's no use case for that. If you're a programmer just using a library, use pkg_resources just like I suggested. If you're building a packaging tool, you should know where to put your hands, and that's it.

Answer 2

The zipimporter used to load a module can be accessed using the __loader__ attribute on the module, so accessing a file within the egg should be as simple as:

__loader__.get_data('path/within/the/egg')