Reversing from module import *

Question

I have a codebase where I'm cleaning up some messy decisions by the previous developer. Frequently, he has done something like:

from scipy import *
from numpy import *

...This, of course, pollutes the name space and makes it difficult to tell where an attribute in the module is originally from.

Is there any way to have Python analyze and fix this for me? Has anyone made a utility for this? If not, how might a utility like this be made?

Answer 1

I think PurityLake's and Martijn Pieters's assisted-manual solutions are probably the best way to go. But it's not impossible to do this programmatically.

First, you need to get a list of all names that existing in the module's dictionary that might be used in the code. I'm assuming your code isn't directly calling any dunder functions, etc.

Then, you need to iterate through them, using inspect.getmodule() to find out which module each object was originally defined in. And I'm assuming that you're not using anything that's been doubly from foo import *-ed. Make a list of all of the names that were defined in the numpy and scipy modules.

Now you can take that output and just replace each foo with numpy.foo.

So, putting it together, something like this:

for modname in sys.argv[1:]:
    with open(modname + '.py') as srcfile:
        src = srcfile.read()
    src = src.replace('from numpy import *', 'import numpy')
    src = src.replace('from scipy import *', 'import scipy')
    mod = __import__(modname)
    for name in dir(mod):
        original_mod = inspect.getmodule(getattr(mod, name))
        if original_mod.__name__ == 'numpy':
            src = src.replace(name, 'numpy.'+name)
        elif original_mod.__name__ == 'scipy':
            src = src.replace(name, 'scipy.'+name)
    with open(modname + '.tmp') as dstfile:
        dstfile.write(src)
    os.rename(modname + '.py', modname + '.bak')
    os.rename(modname + '.tmp', modname + '.py')

If either of the assumptions is wrong, it's not hard to change the code. Also, you might want to use tempfile.NamedTemporaryFile and other improvements to make sure you don't accidentally overwrite things with temporary files. (I just didn't want to deal with the headache of writing something cross-platform; if you're not running on Windows, it's easy.) And add in some error handling, obviously, and probably some reporting.