Making a copy of an entire namespace?

Question

I'd like to make a copy of an entire namespace while replacing some functions with dynamically constructed versions.

In other words, starting with namespace (import tensorflow as tf), I want to make a copy of it, replace some functions with my own versions, and update __globals__ of all the symbols to stay within the new namespace. This needs to be done in topological order of dependency.

I started doing something like it here but now I'm starting to wonder if I'm reinventing the wheel. Care is needed to deal with circular dependencies in system modules, functions/types/objects need to be updated differently, etc.

Can anyone point to existing code that solves a similar task?

Answer 1

Instead of trying to make a copy of the contents of a module and patch everything in it to use the correct globals, you could trick Python into importing everything you want to copy a second time. This will give you a newly initialized copy of all modules, so it won't copy any global state the modules might have (not sure whether you would need that).

import importlib
import sys

def new_module_instances(module_names):
    old_modules = {}
    for name in module_names:
        old_modules[name] = sys.modules.pop(name)
    new_modules = {}
    for name in module_names:
        new_modules[name] = importlib.import_module(name)
    sys.modules.update(old_modules)
    return new_modules

Note that we first delete all modules we want to replace from sys.modules, so they all get import a second time, and the dependencies between these modules are set up correctly automatically. At the end of the function, we restore the original state of sys.modules, so everything else continues to see the original versions of these modules.

Here's an example:

>>> import logging.handlers
>>> new_modules = new_module_instances(['logging', 'logging.handlers'])
>>> logging_clone = new_modules['logging']
>>> logging
<module 'logging' from '/usr/lib/python2.7/logging/__init__.pyc'>
>>> logging_clone
<module 'logging' from '/usr/lib/python2.7/logging/__init__.pyc'>
>>> logging is logging_clone
False
>>> logging is logging.handlers.logging
True
>>> logging_clone is logging_clone.handlers.logging
True

The last three expressions show that the two versions of logging are different modules, and both versions of the handlers module use the correct version of the logging module.

Answer 2

To my mind, you can do this easily:

import imp, string

st = imp.load_module('st', *imp.find_module('string')) # copy the module

def my_upper(a):
    return "a" + a

def my_lower(a):
    return a + "a"

st.upper = my_upper
st.lower = my_lower


print string.upper("hello") # HELLO
print string.lower("hello") # hello

print st.upper("hello") # ahello
print st.lower("hello") # helloa

And when you call st.upper("hello"), it will result in "hello".

So, you don't really need to mess with globals.