Python: Dynamically import module's code from string with importlib

Question

I wish to dynamically import a module in Python (3.7), whereby the module's code is defined within a string.

Below is a working example that uses the imp module, which is deprecated in favour of importlib (as of version 3.4):

import imp

def import_code(code, name):
    # create blank module
    module = imp.new_module(name)
    # populate the module with code
    exec(code, module.__dict__)
    return module

code = """
def testFunc():
    print('spam!')
"""

m = import_code(code, 'test')
m.testFunc()

Python's documentation states that importlib.util.module_from_spec() should be used instead of imp.new_module(). However, there doesn't seem to be a way to create a blank module object using the importlib module, like I could with imp.

How can I use importlib instead of imp to achieve the same result?

Answer 1

You can simply instantiate types.Module:

import types
mod = types.ModuleType("mod")

Then you can populate it with exec just like you did:

exec(code, mod.__dict__)
mod.testFunc() # will print 'spam!'

So your code will look like this:

import types

def import_code(code, name):
    # create blank module
    module = types.ModuleType(name)
    # populate the module with code
    exec(code, module.__dict__)
    return module

code = """
def testFunc():
    print('spam!')
"""

m = import_code(code, 'test')
m.testFunc()

---

As commented by @Error - Syntactical Remorse, you should keep in mind that exec basically executes whatever code is contained in the string you give it, so you should use it with extra care. At least check what you're given, but it'd be good to use exclusively predefined strings.

Answer 2

According to Python documentation module_from_spec()

importlib.util.module_from_spec(spec)

...

This function is preferred over using types.ModuleType to create a new module as spec is used to set as many import-controlled attributes on the module as possible.

Here is what I came up with to load the module from source code located in github repo. It is a way without writing the file to disk.

import requests
url = "https://github.com/udacity/deep-learning-v2-pytorch/raw/master/intro-to-pytorch/helper.py"
r = requests.get(url)

import importlib.util
spec = importlib.util.spec_from_loader('helper', loader=None, origin=url)
helper = importlib.util.module_from_spec(spec)
exec(r.content, helper.__dict__)

helper.view_classify() # executes function from github file