How to make modules from exec("import xxx") in a Python function available?

Question

I have written myself a simple function that sieves through my Python folder and look for where a possible module is. What I want to do is simple. I pass a string of module import and the function will find the folder of the module, cd there, and import it to whichever environment I am working in, e.g.:

anyimport('from fun_abc import *')

Originally I tried:

class anyimport(object):
    def __init__(self, importmodule, pythonpath='/home/user/Python', finddir=finddir):

        ##################################################################    
        ### A BUNCH OF CODES SCANNING THE DIRECTORY AND LOCATE THE ONE ###
        ##################################################################
        ### "pointdir" is where the directory of the file is ###
        ### "evalstr" is a string that looks like this : ---
        ### 'from yourmodule import *'

        os.chdir(pointdir)
        exec evalstr

As I coded the whole thing in iPython Notebook, it works. So the problem slipped by me. Then I found it does not work properly because the modules the function imports stay in the function's local variable space.

Then I found this Stack Overflow discussion "In Python, why doesn't an import in an exec in a function work?". Thus I changed the code to the following:

class anyimport(object):
    def __init__(self, importmodule, pythonpath='/home/user/Python', finddir=finddir):

        ##################################################################    
        ### A BUNCH OF CODES SCANNING THE DIRECTORY AND LOCATE THE ONE ###
        ##################################################################
        ### "pointdir" is where the directory of the file is ###
        ### "evalstr" is a string that looks like this : ---
        ### 'from yourmodule import *'

        sys.path.append(os.path.join(os.path.dirname(__file__), pointdir))
        exec (evalstr, globals())

It still does not work. The function runs without error but the modules are not available to me, say if I run script.py in which I do anyimport('from fun_abc import *') but nothing from fun_abc is there. Python will tell me "NameError: name 'fun_you_want' is not defined".

Would anyone be kind enough to point me in the right direction as to how to solve this problem?

Thank you for your attention and I really appreciate your help!

Attention:

In addition to @Noya's spot-on answer that one has to pass the scope to make the exec work, to avoid "ImportError", you need to add this line before running exec:

sys.path.append(os.path.join(os.path.dirname(__file__), pointdir))        
exec (evalstr, scope)

This is due to the reason that our modification of sys.path assumes the current working directory is always in main/. We need to add the parent directory to sys.path. See this Stack Overflow discussion "ImportError: No module named - Python" for more information on resolving this issue.

Answer 1

exec executes code in the current scope. Inside a function, this means the (function-) local scope.

You can tell exec to put variables in another scope by giving it a tuple (code, scope). For example, you can use globals() to make names available at the module level.

Be aware, that globals

is always the dictionary of the current module (inside a function or method, this is the module where it is defined, not the module from which it is called).

Thus, in your example, you have to pass the desired scope to your utility function:

anyimport.py:

class anyimport(object):
    def __init__(self, importmodule, scope):
        exec (importmodule, scope)

test.py:

a = 42
b = 'foo'

main.py:

from anyimport import anyimport

if __name__ == '__main__':
    anyimport('from test import *', globals())
    # 42 foo
    print a, b

Test it with python main.py. Make sure all files are present in the current directory.

Alternative Solution

If you are not bound to use exec, a more elegant way would be to use the import utilities provided by Python.

The following, taken from https://stackoverflow.com/a/4526709/453074 , is equivalent to from some.package import *:

[...] it's more convenient to user importlib:

globals().update(importlib.import_module('some.package').__dict__) 

.

Answer 2

You might want to try something like this:

_globals = {}
code = """import math;"""
code += """import numpy;"""
code = compile(code, '<string>', 'exec')
exec code in _globals

It's safer than just doing an exec, and it should import correctly inside a function's local scope.

You can then update globals() with whatever modules (or functions) you import.

When using exec for functions, you can get a handle to globals with g = globals(), then do an update on g. For modules, you should do another step... you will want to also update the modules in sys.modules as well.

UPDATE: to be explicit:

>>> def foo(import_string):
...   _globals = {}
...   code = compile(import_string, '<string>', 'exec')
...   exec code in _globals
...   import sys
...   g = globals()
...   g.update(_globals)
...   sys.modules.update(_globals)
... 
>>> foo('import numpy as np')
>>> np.linspace
<function linspace at 0x1009fc848>