mypy importlib module functions

Question

I am using importlib to import modules at runtime. These modules are plugins for my application and must implement 1 or more module-level functions. I have started adding type annotations to my applications and I get an error from mypy stating

Module has no attribute "generate_configuration"

where "generate_configuration" is one of the module functions.

In this example, the module is only required to have a generate_configuration function in it. The function takes a single dict argument.

def generate_configuration(data: Dict[str, DataFrame]) -> None: ...

I have been searching around for how to specify the interface of a module but all I can find are class interfaces. Can someone point me to some documentation showing how to do this? My google-fu is failing me on this one.

The code that loads this module is shown below. The error is generated by the last line.

plugin_directory = os.path.join(os.path.abspath(directory), 'Configuration-Generation-Plugins')
plugins = (
    module_file
    for module_file in Path(plugin_directory).glob('*.py')
)
sys.path.insert(0, plugin_directory)
for plugin in plugins:
    plugin_module = import_module(plugin.stem)
    plugin_module.generate_configuration(directory, points_list)

Answer 1

The type annotation for importlib.import_module simply returns types.ModuleType

From the typeshed source:

def import_module(name: str, package: Optional[str] = ...) -> types.ModuleType: ...

This means that the revealed type of plugin_module is Module -- which doesn't have your specific attributes.

Since mypy is a static analysis tool, it can't know that the return value of that import has a specific interface.

Here's my suggestion:

1. Make a type interface for your module (it doesn't have to be instantiated, it'll just help mypy figure things out)

   class ModuleInterface:
       @staticmethod
       def generate_configuration(data: Dict[str, DataFrame]) -> None: ...
   

2. Make a function which imports your module, you may need to sprinkle # type: ignore, though if you use __import__ instead of import_module you may be able to avoid this limitation

   def import_module_with_interface(modname: str) -> ModuleInterface:
       return __import__(modname, fromlist=['_trash'])  # might need to ignore the type here
   

3. Enjoy the types :)

The sample code I used to verify this idea:

class ModuleInterface:
    @staticmethod
    def compute_foo(bar: str) -> str: ...


def import_module_with_interface(modname: str) -> ModuleInterface:
    return __import__(modname, fromlist=['_trash'])


def myf() -> None:
    mod = import_module_with_interface('test2')
    # mod.compute_foo()  # test.py:12: error: Too few arguments for "compute_foo" of "ModuleInterface"
    mod.compute_foo('hi')