Dynamic loading of uncompiled python plugins in py2exe compile code

Question

My Python application is constructed as such that some functionality is available as plugins. The plugin architecture currently is very simple: I have a plugins folder/package which contains some python modules. I load the relevant plugin as follows:

plugin_name = blablabla
try:
    module = __import__(plugin_name, fromlist='do_something')
except ImportError:
   #some error handling ...

and then execute:

try:
    loans = module.do_something(id_t, pin_t)
except xxx:
    # error handling

I compile the application to a Windows binary using py2exe. This works fine, except for that the fact that all plugins are (and have to be) included in the binary. This is not very practical, since for each new plugin, I have to recompile and release a new version of my application. It would be better if a new plugin (i.e. python file) could be copied to some application plugin folder, and that the Python code in the file code be interpreted on-the-fly by my application.

What is the best approach to do so?

(I've though of reading each line of the selected plugin file, and applying an exec statement to it. But there might be better ways...)

Answer 1

If you don't mind that plugin's will be release as .py files, you can do something like the following. Put all your plugin's under a "plugin" subdir, and create an empty "__init__.py". Doing runtime, it will import the package along with all the modules in that directory. Check Dive In Python for explanation... but here's what I finally end up using.

def load_plugin(path):
    import imp
    """Load plugin from directory and return list of modules"""
    files = os.listdir( path )
    test = re.compile(".py$", re.IGNORECASE)          
    files = filter(test.search, files)                     
    filenameToModuleName = lambda f: os.path.splitext(f)[0]
    moduleNames = sorted(map(filenameToModuleName, files))
    f, filename, desc = imp.find_module('plugin')
    plugin = imp.load_module('plugin', f, filename, desc)
    modules = []

    #print moduleNames
    for m in moduleNames:
        # skip any files starting with '__', such as __init__.py
        if m.startswith('__'):
            continue
        try:
            f, filename, desc = imp.find_module(m, plugin.__path__)
            modules.append( imp.load_module(m, f, filename, desc))
        except ImportError:
            continue
    return modules