Python import one subpackage without others

Question

I have the following package structure

package
    __init__.py
    sub1
        __init__.py
        foo.py      # Contains class Foo
    sub2
        __init__.py
        bar.py      # Contains class Bar

I want to be able to just import package and have package.Foo and package.Bar, i.e. I want to have the subpackages be transparent to users.

The catch is that importing sub2 takes a long time, and many users don't care at all about the stuff in sub2 and only want the stuff in sub1. Thus I want users to be able to say import package.sub1 or from package import sub1 to just import sub1 and skip the import of sub2.

I know I can achieve the first part by having package/__init__.py contain

from .sub1 import *
from .sub2 import *

and having package/sub1/__init__.py be from .foo import Foo and similarly for sub2. However, this will always import sub1 and sub2 even if the user tries to import only package.sub1.

Correspondingly, I can achieve the second part by having package/__init__.py be empty and using the same sub1/__init__.py as above. However, then just saying import package doesn't load sub1 or sub2, so users would have to explicitly load them and then refer to package.sub1.Foo.

Ideally a solution would work both in 2.7.10 and 3.5.0, but I'll accept one or the other if both isn't possible.

Answer 1

The LazyLoader class is provided for exactly this kind of situation: postponing loading of the module when it is actually used, instead of at the point of importing it.

To build a lazy loader you can follow the example in the documentation:

suffixes = importlib.machinery.SOURCE_SUFFIXES
loader = importlib.machinery.SourceFileLoader
lazy_loader = importlib.util.LazyLoader.factory(loader)
finder = importlib.machinery.FileFinder(path, [(lazy_loader, suffixes)])

then you can use finder.find_spec to obtain the spec of a module and pass the result to Loader.create_module to load it.

This is a bit cumbersome to do manually for just one module.

Note that searching for "lazy import python" you'll find quite a few solutions that have different pro and cons, some of which run in python2.x. However the LazyLoader class above is the official way of doing it in python3.5+