What is the purpose of the sub-interpreter API in CPython?

Question

I'm unclear on why the sub-interpreter API exists and why it's used in modules such as the mod_wsgi apache module. Is it mainly used for creating a security sandbox for different applications running within the same process, or is it a way to allow concurrency with multiple threads? Maybe both? Are there other purposes?

Answer 1

I imagine the purpose is to create separate python execution environments. For instance, mod_wsgi (Apache Python module) hosts a single python interpreter and then hosts multiple applications within sub-interpreters (in the default configuration).

Some key points from the documentation:

• This is an (almost) totally separate environment for the execution of Python code. In particular, the new interpreter has separate, independent versions of all imported modules, including the fundamental modules __builtin__, __main__ and sys.
• The table of loaded modules (sys.modules) and the module search path (sys.path) are also separate.
• Because sub-interpreters (and the main interpreter) are part of the same process, the insulation between them isn’t perfect — for example, using low-level file operations like os.close() they can (accidentally or maliciously) affect each other’s open files.
• Because of the way extensions are shared between (sub-)interpreters, some extensions may not work properly; this is especially likely when the extension makes use of (static) global variables, or when the extension manipulates its module’s dictionary after its initialization.