How to mimic Python modules without import?

Question

I’ve tried to develop a « module expander » tool for Python 3 but I've some issues.

The idea is the following : for a given Python script main.py, the tool generates a functionally equivalent Python script expanded_main.py, by replacing each import statement by the actual code of the imported module; this assumes that the Python source code of the imported is accessible. To do the job the right way, I’m using the builtin module ast of Python as well as astor, a third-party tool allowing to dump the AST back into Python source. The motivation of this import expander is to be able to compile a script into one single bytecode chunk, so the Python VM should not take care of importing modules (this could be useful for MicroPython, for instance).

The simplest case is the statement:

from import my_module1 import *

To transform this, my tool looks for a file my_module1.py and it replaces the import statement by the content of this file. Then, the expanded_main.py can access any name defined in my_module, as if the module was imported the normal way. I don’t care about subtle side effects that may reveal the trick. Also, to simplify, I treat from import my_module1 import a, b, c as the previous import (with asterisk), without caring about possible side effect. So far so good.

Now here is my point. How could you handle this flavor of import:

import my_module2

My first idea was to mimic this by creating a class having the same name as the module and copying the content of the Python file indented:

class my_module2:
    # content of my_module2.py
    …

This actually works for many cases but, sadly, I discovered that this has several glitches: one of these is that it fails with functions having a body referring to a global variable defined in the module. For example, consider the following two Python files:

# my_module2.py 
g = "Hello"
def greetings():
    print (g + " World!")

and

# main.py
import my_module2
print(my_module2.g)
my_module2.greetings()

At execution, main.py prints "Hello" and "Hello World!". Now, my expander tool shall generate this:

# expanded_main.py
class my_module2:
    g = "Hello"
    def greetings():
        print (g + " World!")
print(my_module2.g)
my_module2.greetings()

At execution of expanded_main.py, the first print statement is OK ("Hello") but the greetings function raises an exception: NameError: name 'g' is not defined. What happens actually is that

• in the module my_module2, g is a global variable,
• in the class my_module2, g is a class variable, which should be referred as my_module2.g.

Other similar side effects happens when you define functions, classes, … in my_module2.py and you want to refer to them in other functions, classes, … of the same my_module2.py.

Any idea how these problems could be solved?

Apart classes, are there other Python constructs that allow to mimic a module?

Final note: I’m aware that the tool should take care 1° of nested imports (recursion), 2° of possible multiple import of the same module. I don't expect to discuss these topics here.

Answer 1

You can execute the source code of a module in the scope of a function, specifically an instance method. The attributes can then be made available by defining __getattr__ on the corresponding class and keeping a copy of the initial function's locals(). Here is some sample code:

class Importer:
    def __init__(self):

        g = "Hello"

        def greetings():
            print (g + " World!")

        self._attributes = locals()

    def __getattr__(self, item):
        return self._attributes[item]


module1 = Importer()
print(module1.g)
module1.greetings()

Nested imports are handled naturally by replacing them the same way with an instance of Importer. Duplicate imports shouldn't be a problem either.