Proper way to dynamically import a module with relative imports?

Question

I need to dynamically import modules into my project from another package.

The structure is like:

project_folder/
    project/
        __init__.py
        __main__.py
    plugins/
        __init__.py
        plugin1/
            __init__.py
            ...
        plugin2/
            __init__.py
            ...

I made this function to load a module:

import os

from importlib.util import spec_from_file_location, module_from_spec


def load_module(path, name=""):
    """ loads a module by path """
    try:
        name = name if name != "" else path.split(os.sep)[-1]  # take the module name by default
        spec = spec_from_file_location(name, os.path.join(path, "__init__.py"))
        plugin_module = module_from_spec(spec)
        spec.loader.exec_module(plugin_module)
        return plugin_module
    except Exception as e:
        print("failed to load module", path, "-->", e)

It works, unless the module uses relative imports:

failed to load module /path/to/plugins/plugin1 --> Parent module 'plugin1' not loaded, cannot perform relative import

What am I doing wrong?

Answer 1

I managed to solve my own issue after a LOT of googling. Turns out I needed to import using relative paths:

>>> from importlib import import_module
>>> config = import_module("plugins.config")
>>> config
<module 'plugins.config' from '/path/to/plugins/config/__init__.py'>
>>> 

Answer 2

I had a similar problem not long ago. I added the path of the project folder to the sys.path using the module's absolute path like this:

import sys
import os
sys.path.append(os.path.dirname(os.path.realpath(__file__))+'/..')

This adds the project_folder to the sys.path thus allowing the import statement to find the plugin modules.