Python setup, install one module as a sub module of another module?

Question

I would like to create a package that installs into a submodule of another python package as an extension.

Basically the original module is set up like so:

origmodule/
    __init__.py
    stuff.py
    ext/
        __init__.py
        # This module is empty

And then have my extension module installed into the empty origmodule.ext module. Reading the instructions for distutils it was not clear if this is possible or supported. The end goal is so that my extension module once installed would be imported like this:

import origmodule.ext.my_extension

Answer 1

You cannot do that with setuptools, installing inside another package is not supported.

Instead, you want to use entry points instead. Your origmodule should look for anything registered as an entry point for a specific key, and your extension module should register itself for that key.

Your extension registers an extension point:

entry_points={
    'some.opaque.string.unique.to.origmodule':
        ['my_extension = my.extension.package:some.entry.point', ]
}

which your origmodule then can discover by asking pkg_resources:

import pkg_resources

for entrypoint in pkg_resources.iter_entry_points(group='some.opaque.string.unique.to.origmodule'):
    extension = entrypoint.load()

Take a look at the Babel project as an example. Babel can extract translatable text from files; it knows how to do this for Python source code, but it also supports extensions that can extract such text from other formats.

Such extraction methods can be registered as extension points. Babel documents this in the Writing Extraction Methods section. The message extraction code then loads those entry points when extracting text:

GROUP_NAME = 'babel.extractors'

# ...

for entry_point in working_set.iter_entry_points(GROUP_NAME,
                                                 method):
    func = entry_point.load(require=True)
    break

The Mako templating language supplies such a plugin; it registers the entry point, which points to the actual implementation:

[babel.extractors]
mako = mako.ext.babelplugin:extract

def extract(fileobj, keywords, comment_tags, options):
    # ...