Menu
As far as I understand, a python module is never imported twice, i.e. the code in the module only gets executed the first time it is imported. Subsequent import statements just add the module to the scope of the import.
I have a module called "TiledConvC3D.py" that seems to be imported multiple times though. I use pdb to print the stack at the top of the code for this module.
Here is the end of the stack trace from the first time the module is executed:
File "<anonymized>/python_modules/Theano/theano/gof/cmodule.py", line 328, in refresh
key = cPickle.load(open(key_pkl, 'rb'))
File "<anonymized>/ops/TiledConvG3D.py", line 565, in <module>
import TiledConvC3D
File "<anonymized>/ops/TiledConvC3D.py", line 18, in <module>
pdb.traceback.print_stack()
It goes on to be executed several more times. However, the complete stack trace for the second time it is called does not show any calls to reload, so these executions should not be occurring:
File "sup_train_conj_grad.py", line 103, in <module>
dataset = Config.get_dataset(dataset_node)
File "<anonymized>/Config.py", line 279, in get_dataset
from datasets import NewWiskott
File "<anonymized>/datasets/NewWiskott.py", line 16, in <module>
normalizer_train = video.ContrastNormalizer3D(sigma, global_per_frame = False, input_is_5d = True)
File "<anonymized>/util/video.py", line 204, in __init__
self.f = theano.function([input],output)
File "<anonymized>/python_modules/Theano/theano/compile/function.py", line 105, in function
allow_input_downcast=allow_input_downcast)
File "<anonymized>/python_modules/Theano/theano/compile/pfunc.py", line 270, in pfunc
accept_inplace=accept_inplace, name=name)
File "<anonymized>/python_modules/Theano/theano/compile/function_module.py", line 1105, in orig_function
fn = Maker(inputs, outputs, mode, accept_inplace = accept_inplace).create(defaults)
File "/u/goodfeli/python_modules/Theano/theano/compile/function_module.py", line 982, in create
_fn, _i, _o = self.linker.make_thunk(input_storage = input_storage_lists)
File "<anonymized>/python_modules/Theano/theano/gof/link.py", line 321, in make_thunk
output_storage = output_storage)[:3]
File "<anonymized>/python_modules/Theano/theano/gof/cc.py", line 1178, in make_all
output_storage = node_output_storage)
File "<anonymized>/python_modules/Theano/theano/gof/cc.py", line 774, in make_thunk
cthunk, in_storage, out_storage, error_storage = self.__compile__(input_storage, output_storage)
File "<anonymized>/python_modules/Theano/theano/gof/cc.py", line 723, in __compile__
output_storage)
File "<anonymized>/python_modules/Theano/theano/gof/cc.py", line 1037, in cthunk_factory
module = get_module_cache().module_from_key(key=key, fn=self.compile_cmodule)
File "<anonymized>/python_modules/Theano/theano/gof/cc.py", line 59, in get_module_cache
return cmodule.get_module_cache(config.compiledir)
File "<anonymized>/python_modules/Theano/theano/gof/cmodule.py", line 576, in get_module_cache
_module_cache = ModuleCache(dirname, force_fresh=force_fresh)
File "<anonymized>/python_modules/Theano/theano/gof/cmodule.py", line 268, in __init__
self.refresh()
File "<anonymized>/python_modules/Theano/theano/gof/cmodule.py", line 326, in refresh
key = cPickle.load(open(key_pkl, 'rb'))
File "<anonymized>/ops/TiledConvV3D.py", line 504, in <module>
import TiledConvG3D
File "<anonymized>/ops/TiledConvG3D.py", line 565, in <module>
import TiledConvC3D
File "<anonymized>/ops/TiledConvC3D.py", line 22, in <module>
pdb.traceback.print_stack()
Moreover, I also check the id of __builtin__.__import__ . At the very start of my main script, I import __builtin__ and print id(__builtin__.__import__) before doing any other imports. I also print id(__builtin__.import__) from inside my module that is being imported multiple times, and the value of the id does not change.
Are there other mechanisms besides calling reload and overriding __builtin__.__import__ that could explain my module getting loaded multiple times?
775
The rules are quite simple: the same module is evaluated only once, in other words, the module-level scope is executed just once. If the module, once evaluated, is imported again, it's second evaluation is skipped and the resolved already exports are used.
Yes, you can import module as many times as you want in one Python program, no matter what module it is. Every subsequent import after the first accesses the cached module instead of re-evaluating it. And you can see the cached modules in sys.
What happens if a module is imported twice? The module is only loaded the first time the import statement is executed and there is no performance loss by importing it again.
A Python module can be imported twice if the module is found twice in the path. For example, say your project is laid out like so:
Suppose your PYTHONPATH (sys.path) includes src and src/package1:
PYTHONPATH=/path/to/src:/path/to/src/package1 If that's the case, you can import the same module twice like this:
from package1 import spam import spam And Python will think they are different modules. Is that what's going on?
Also, per the discussion below (for users searching this question), another way a module can be imported twice is if there is an exception midway through the first import. For example, if spam imports eggs, but importing eggs results in an exception inside the module, it can be imported again.
176
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
