Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Dynamic importing of modules followed by instantiation of objects with a certain baseclass from said modules

Tags:

python

plugins

I'm writing an application. No fancy GUI:s or anything, just a plain old console application. This application, lets call it App, needs to be able to load plugins on startup. So, naturally, i created a class for the plugins to inherit from:

class PluginBase(object):
    def on_load(self):
        pass
    def on_unload(self):
        pass
    def do_work(self, data):
        pass

The idea being that on startup, App would walk through the current dir, including subdirs, searching for modules containing classes that themselves are subclasses of PluginBase.

More code:

class PluginLoader(object):
    def __init__(self, path, cls):
        """ path=path to search (unused atm), cls=baseclass """
        self.path=path
    def search(self):
        for root, dirs, files in os.walk('.'):
            candidates = [fname for fname in files if fname.endswith('.py') \
                                    and not fname.startswith('__')]
        ## this only works if the modules happen to be in the current working dir
        ## that is not important now, i'll fix that later
        if candidates:
            basename = os.path.split(os.getcwd())[1]
            for c in candidates:
                modname = os.path.splitext(c)[0]
                modname = '{0}.{1}'.format(basename, modname)
                __import__(mod)
                module = sys.modules[mod]

After that last line in search I'd like to somehow a) find all classes in the newly loaded module, b) check if one or more of those classes are subclasses of PluginBase and c) (if b) instantiate that/those classes and add to App's list of loaded modules.

I've tried various combinations of issubclass and others, followed by a period of intense dir:ing and about an hour of panicked googling. I did find a similar approach to mine here and I tried just copy-pasting that but got an error saying that Python doesn't support imports by filename, at which point I kind of lost my concentration and as a result of that, this post was written.

I'm at my wits end here, all help appreciated.

like image 858
manneorama Avatar asked Jan 24 '11 21:01

manneorama


People also ask

How do you dynamically import a module using a function?

To load dynamically a module call import(path) as a function with an argument indicating the specifier (aka path) to a module. const module = await import(path) returns a promise that resolves to an object containing the components of the imported module. } = await import(path);

How do I import a dynamic module in Python?

Use importlib. import_module() to dynamically import a module Call importlib. import_module(module_name) to import the module with module_name into the current program.

How do you instantiate an object in Python?

In short, Python's instantiation process starts with a call to the class constructor, which triggers the instance creator, . __new__() , to create a new empty object. The process continues with the instance initializer, . __init__() , which takes the constructor's arguments to initialize the newly created object.

How do you create an instance of a class dynamically in Python?

Classes can be dynamically created using the type() function in Python. The type() function is used to return the type of the object. The above syntax returns the type of object. print ( type ( "Geeks4Geeks !" ))


2 Answers

You might do something like this:

for c in candidates:
    modname = os.path.splitext(c)[0]
    try:
        module=__import__(modname)   #<-- You can get the module this way
    except (ImportError,NotImplementedError):
        continue
    for cls in dir(module):          #<-- Loop over all objects in the module's namespace
        cls=getattr(module,cls)
        if (inspect.isclass(cls)                # Make sure it is a class 
            and inspect.getmodule(cls)==module  # Make sure it was defined in module, not just imported
            and issubclass(cls,base)):          # Make sure it is a subclass of base
            # print('found in {f}: {c}'.format(f=module.__name__,c=cls))
            classList.append(cls)

To test the above, I had to modify your code a bit; below is the full script.

import sys
import inspect
import os

class PluginBase(object): pass

def search(base):
    for root, dirs, files in os.walk('.'):
        candidates = [fname for fname in files if fname.endswith('.py') 
                      and not fname.startswith('__')]
        classList=[]
        if candidates:
            for c in candidates:
                modname = os.path.splitext(c)[0]
                try:
                    module=__import__(modname)
                except (ImportError,NotImplementedError):
                    continue
                for cls in dir(module):
                    cls=getattr(module,cls)
                    if (inspect.isclass(cls)
                        and inspect.getmodule(cls)==module
                        and issubclass(cls,base)):
                        # print('found in {f}: {c}'.format(f=module.__name__,c=cls))
                        classList.append(cls)
        print(classList)

search(PluginBase)
like image 180
unutbu Avatar answered Oct 17 '22 03:10

unutbu


You would make this a lot easier if you forced some constraints on the plugin writer, for example that all plugins must be packages that contain a load_plugin( app, config) function that returns a Plugin instance. Then all you have to do is try to import these packages and run the function.

like image 20
Jochen Ritzel Avatar answered Oct 17 '22 03:10

Jochen Ritzel