What could cause a python module to be imported twice?

Question

As far as I understand, a python module is never imported twice, i.e. the code in the module only gets executed the first time it is imported. Subsequent import statements just add the module to the scope of the import.

I have a module called "TiledConvC3D.py" that seems to be imported multiple times though. I use pdb to print the stack at the top of the code for this module.

Here is the end of the stack trace from the first time the module is executed:

File "<anonymized>/python_modules/Theano/theano/gof/cmodule.py", line 328, in refresh
  key = cPickle.load(open(key_pkl, 'rb'))
File "<anonymized>/ops/TiledConvG3D.py", line 565, in <module>
  import TiledConvC3D
File "<anonymized>/ops/TiledConvC3D.py", line 18, in <module>
  pdb.traceback.print_stack()

It goes on to be executed several more times. However, the complete stack trace for the second time it is called does not show any calls to reload, so these executions should not be occurring:

File "sup_train_conj_grad.py", line 103, in <module>
  dataset = Config.get_dataset(dataset_node)
File "<anonymized>/Config.py", line 279, in get_dataset
  from datasets import NewWiskott
File "<anonymized>/datasets/NewWiskott.py", line 16, in <module>
  normalizer_train = video.ContrastNormalizer3D(sigma, global_per_frame = False, input_is_5d = True)
File "<anonymized>/util/video.py", line 204, in __init__
  self.f = theano.function([input],output)
File "<anonymized>/python_modules/Theano/theano/compile/function.py", line 105, in function
  allow_input_downcast=allow_input_downcast)
File "<anonymized>/python_modules/Theano/theano/compile/pfunc.py", line 270, in pfunc
  accept_inplace=accept_inplace, name=name)
File "<anonymized>/python_modules/Theano/theano/compile/function_module.py", line 1105, in orig_function
  fn = Maker(inputs, outputs, mode, accept_inplace = accept_inplace).create(defaults)
File "/u/goodfeli/python_modules/Theano/theano/compile/function_module.py", line 982, in create
  _fn, _i, _o = self.linker.make_thunk(input_storage = input_storage_lists)
File "<anonymized>/python_modules/Theano/theano/gof/link.py", line 321, in make_thunk
  output_storage = output_storage)[:3]
File "<anonymized>/python_modules/Theano/theano/gof/cc.py", line 1178, in make_all
  output_storage = node_output_storage)
File "<anonymized>/python_modules/Theano/theano/gof/cc.py", line 774, in make_thunk
  cthunk, in_storage, out_storage, error_storage = self.__compile__(input_storage, output_storage)
File "<anonymized>/python_modules/Theano/theano/gof/cc.py", line 723, in __compile__
  output_storage)
File "<anonymized>/python_modules/Theano/theano/gof/cc.py", line 1037, in cthunk_factory
  module = get_module_cache().module_from_key(key=key, fn=self.compile_cmodule)
File "<anonymized>/python_modules/Theano/theano/gof/cc.py", line 59, in get_module_cache
  return cmodule.get_module_cache(config.compiledir)
File "<anonymized>/python_modules/Theano/theano/gof/cmodule.py", line 576, in get_module_cache
  _module_cache = ModuleCache(dirname, force_fresh=force_fresh)
File "<anonymized>/python_modules/Theano/theano/gof/cmodule.py", line 268, in __init__
  self.refresh()
File "<anonymized>/python_modules/Theano/theano/gof/cmodule.py", line 326, in refresh
  key = cPickle.load(open(key_pkl, 'rb'))
File "<anonymized>/ops/TiledConvV3D.py", line 504, in <module>
  import TiledConvG3D
File "<anonymized>/ops/TiledConvG3D.py", line 565, in <module>
  import TiledConvC3D
File "<anonymized>/ops/TiledConvC3D.py", line 22, in <module>
  pdb.traceback.print_stack()

Moreover, I also check the id of __builtin__.__import__ . At the very start of my main script, I import __builtin__ and print id(__builtin__.__import__) before doing any other imports. I also print id(__builtin__.import__) from inside my module that is being imported multiple times, and the value of the id does not change.

Are there other mechanisms besides calling reload and overriding __builtin__.__import__ that could explain my module getting loaded multiple times?

Answer 1

A Python module can be imported twice if the module is found twice in the path. For example, say your project is laid out like so:

• src/
  • package1/
    • spam.py
    • eggs.py

Suppose your PYTHONPATH (sys.path) includes src and src/package1:

PYTHONPATH=/path/to/src:/path/to/src/package1 

If that's the case, you can import the same module twice like this:

from package1 import spam import spam 

And Python will think they are different modules. Is that what's going on?

Also, per the discussion below (for users searching this question), another way a module can be imported twice is if there is an exception midway through the first import. For example, if spam imports eggs, but importing eggs results in an exception inside the module, it can be imported again.