Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why are there dummy modules in sys.modules?

Tags:

python

import

Importing the standard "logging" module pollutes sys.modules with a bunch of dummy entries:

Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit (Intel)] on win32
>>> import sys
>>> import logging
>>> sorted(x for x in sys.modules.keys() if 'log' in x)
['logging', 'logging.atexit', 'logging.cStringIO', 'logging.codecs', 
'logging.os', 'logging.string', 'logging.sys', 'logging.thread', 
'logging.threading', 'logging.time', 'logging.traceback', 'logging.types']

# and perhaps even more surprising:
>>> import traceback
>>> traceback is sys.modules['logging.traceback']
False
>>> sys.modules['logging.traceback'] is None
True

So importing this package puts extra names into sys.modules, except that they are not modules, just references to None. Other modules (e.g. xml.dom and encodings) have this issue as well. Why?

Edit: Building on bobince's answer, there are pages describing the origin (see section "Dummy Entries in sys.modules") and future of the feature.

like image 294
Peter Hansen Avatar asked Dec 24 '09 13:12

Peter Hansen


People also ask

Why would you choose to use the SYS module?

The sys module in Python provides various functions and variables that are used to manipulate different parts of the Python runtime environment. It allows operating on the interpreter as it provides access to the variables and functions that interact strongly with the interpreter.

What is the difference between OS and sys module in Python?

The os module contains two sub-modules os. sys (same as sys) and os. path that are dedicated to the system and directories; respectively. Whenever possible, you should use the functions provided by these modules for file, directory, and path manipulations.

Is sys a Python built in module?

One particular module deserves some attention: sys, which is built into every Python interpreter.

What does Sys path insert do?

sys. path is a built-in variable within the sys module. It contains a list of directories that the interpreter will search in for the required module. When a module(a module is a python file) is imported within a Python file, the interpreter first searches for the specified module among its built-in modules.


1 Answers

None values in sys.modules are cached failures of relative lookups.

So when you're in package foo and you import sys, Python looks first for a foo.sys module, and if that fails goes to the top-level sys module. To avoid having to check the filesystem for foo/sys.py again on further relative imports, it stores None in the sys.modules to flag that the module didn't exist and a subsequent import shouldn't look there again, but go straight to the loaded sys.

This is a cPython implementation detail you can't usefully rely on, but you will need to know it if you're doing nasty magic import/reload hacking.

It happens to all packages, not just logging. For example, import xml.dom and see xml.dom.xml in the module list as it tries to import xml from inside xml.dom.

As Python moves towards absolute import this ugliness will happen less.

like image 84
bobince Avatar answered Sep 22 '22 05:09

bobince