Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is the purpose of the sub-interpreter API in CPython?

I'm unclear on why the sub-interpreter API exists and why it's used in modules such as the mod_wsgi apache module. Is it mainly used for creating a security sandbox for different applications running within the same process, or is it a way to allow concurrency with multiple threads? Maybe both? Are there other purposes?

like image 961
James Whetstone Avatar asked Apr 16 '09 07:04

James Whetstone


1 Answers

I imagine the purpose is to create separate python execution environments. For instance, mod_wsgi (Apache Python module) hosts a single python interpreter and then hosts multiple applications within sub-interpreters (in the default configuration).

Some key points from the documentation:

  • This is an (almost) totally separate environment for the execution of Python code. In particular, the new interpreter has separate, independent versions of all imported modules, including the fundamental modules __builtin__, __main__ and sys.
  • The table of loaded modules (sys.modules) and the module search path (sys.path) are also separate.
  • Because sub-interpreters (and the main interpreter) are part of the same process, the insulation between them isn’t perfect — for example, using low-level file operations like os.close() they can (accidentally or maliciously) affect each other’s open files.
  • Because of the way extensions are shared between (sub-)interpreters, some extensions may not work properly; this is especially likely when the extension makes use of (static) global variables, or when the extension manipulates its module’s dictionary after its initialization.
like image 148
codeape Avatar answered Sep 23 '22 00:09

codeape