How to import all submodules?

Question

I have a directory structure as follows:

| main.py | scripts |--| __init__.py    | script1.py    | script2.py    | script3.py 

From main.py, the module scripts is imported. I tried using pkgutils.walk_packages in combination with __all__, but using that, I can only import all the submodules directly under main using from scripts import *. I would like to get them all under scripts. What would be the cleanest way to import all the submodules of scripts so that I could access scripts.script1 from main?

EDIT: I am sorry that I was a bit vague. I would like to import the submodules on run-time without specifying them explicitly in __init__.py. I can use pkgutils.walk_packages to get the submodule names (unless someone knows of a better way), but I am not sure of the cleanest way to use these names (or maybe the ImpImporters that walk_packages returns?) to import them.

Answer 1

Edit: Here's one way to recursively import everything at runtime...

(Contents of __init__.py in top package directory)

import pkgutil  __all__ = [] for loader, module_name, is_pkg in pkgutil.walk_packages(__path__):     __all__.append(module_name)     _module = loader.find_module(module_name).load_module(module_name)     globals()[module_name] = _module 

I'm not using __import__(__path__+'.'+module_name) here, as it's difficult to properly recursively import packages using it. If you don't have nested sub-packages, and wanted to avoid using globals()[module_name], though, it's one way to do it.

There's probably a better way, but this is the best I can do, anyway.

Original Answer (For context, ignore othwerwise. I misunderstood the question initially):

What does your scripts/__init__.py look like? It should be something like:

import script1 import script2 import script3 __all__ = ['script1', 'script2', 'script3'] 

You could even do without defining __all__, but things (pydoc, if nothing else) will work more cleanly if you define it, even if it's just a list of what you imported.

Answer 2

This is based on the answer that kolypto provided, but his answer does not perform recursive import of packages, whereas this does. Although not required by the main question, I believe recursive import applies and can be very useful in many similar situations. I, for one, found this question when searching on the topic.

This is a nice, clean way of performing the import of the subpackage's modules, and should be portable as well, and it uses the standard lib for python 2.7+ / 3.x.

import importlib import pkgutil   def import_submodules(package, recursive=True):     """ Import all submodules of a module, recursively, including subpackages      :param package: package (name or actual module)     :type package: str | module     :rtype: dict[str, types.ModuleType]     """     if isinstance(package, str):         package = importlib.import_module(package)     results = {}     for loader, name, is_pkg in pkgutil.walk_packages(package.__path__):         full_name = package.__name__ + '.' + name         results[full_name] = importlib.import_module(full_name)         if recursive and is_pkg:             results.update(import_submodules(full_name))     return results 

Usage:

# from main.py, as per the OP's project structure import scripts import_submodules(scripts)  # Alternatively, from scripts.__init__.py import_submodules(__name__)