Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Difference between exec behavior when module is imported or not

I am running to of the following programs. Importantly, imagine that there is mymodule.py file in the directory where both these programs are located.

The first:

exec('''import sys
import os
os.chdir('/') 
sys.path = []
import mymodule''', {})

The second:

import mymodule
exec('''import sys
import os
os.chdir('/') 
sys.path = []
import mymodule''', {})

The first snippet raises ImportError as expected (after all, the directory where mymodule is located is not in path). The second snippet, however, does not, even though mymodule is also not in its path and the environment I am giving it is empty.

My question is why

like image 451
Dmitry Torba Avatar asked Feb 27 '18 19:02

Dmitry Torba


People also ask

What happens when a module is imported?

When a module is first imported, Python searches for the module and if found, it creates a module object 1, initializing it. If the named module cannot be found, a ModuleNotFoundError is raised. Python implements various strategies to search for the named module when the import machinery is invoked.

What is the difference between import module and from module import object?

The difference between import and from import in Python is: import imports the whole library. from import imports a specific member or members of the library.

Does Python import execute code?

When you import a module, Python will run all the code that's in that module. So if your Python file is meant to be imported as a module, be careful not to put side effects at the top-level of your .

What happens if you import a module twice?

The rules are quite simple: the same module is evaluated only once, in other words, the module-level scope is executed just once. If the module, once evaluated, is imported again, it's second evaluation is skipped and the resolved already exports are used.


Video Answer


2 Answers

According to The import system - The module cache,

The first place checked during import search is sys.modules. This mapping serves as a cache of all modules that have been previously imported, including the intermediate paths. So if foo.bar.baz was previously imported, sys.modules will contain entries for foo, foo.bar, and foo.bar.baz. Each key will have as its value the corresponding module object.

During import, the module name is looked up in sys.modules and if present, the associated value is the module satisfying the import, and the process completes. However, if the value is None, then a ModuleNotFoundError is raised. If the module name is missing, Python will continue searching for the module.

The second snippets successfully imports mymodule; it's cached in sys.modules, so no search in other places occurs.

like image 165
falsetru Avatar answered Oct 10 '22 08:10

falsetru


This has nothing to do with exec(), and is a simple misunderstanding about what is available on your sys.path when running a script, and when Python looks for files to load.

You state:

I am running to of the following programs. Importantly, imagine that there is mymodule.py file in the directory where both these programs are located.

[...]

The second snippet, however, does not, even though mymodule is also not in its path

The module is on its path. The directory your script is located in, is added at the start of the module search path. See Command line:

<script>

Execute the Python code contained in script, which must be a filesystem path (absolute or relative) referring to either a Python file, a directory containing a __main__.py file, or a zipfile containing a __main__.py file.

[...]

If the script name refers directly to a Python file, the directory containing that file is added to the start of sys.path, and the file is executed as the __main__ module.

Bold emphasis mine.

So, mymodule.py, which you state is located in the same directory as the scripts you are running, is on the path.

Once loaded, modules stay loaded. import <module> will only look at the module search path if there is not already a module in sys.modules by that name. It doesn't matter if you use exec or not to do the import.

From the import statement documentation:

The basic import statement (no from clause) is executed in two steps:

  1. find a module, loading and initializing it if necessary
  2. define a name or names in the local namespace for the scope where the import statement occurs.

The if necessary part is the important bit.

Further, from The import system:

The import statement combines two operations; it searches for the named module, then it binds the results of that search to a name in the local scope.

[...]

When a module is first imported, Python searches for the module and if found, it creates a module object, initializing it.

and from The module cache:

The first place checked during import search is sys.modules. This mapping serves as a cache of all modules that have been previously imported, including the intermediate paths. So if foo.bar.baz was previously imported, sys.modules will contain entries for foo, foo.bar, and foo.bar.baz. Each key will have as its value the corresponding module object.

During import, the module name is looked up in sys.modules and if present, the associated value is the module satisfying the import, and the process completes.

So by the time your exec() code runs, the first import mymodule had already succeeded and sys.modules[mymodule] exists. The secondimport mymodule` finds that object, and the search ends.

like image 44
Martijn Pieters Avatar answered Oct 10 '22 09:10

Martijn Pieters