same python interpreter instance running multiple scripts simultaneously?

Question

6-7 years ago i saw an initiative of a way to run python on tight resources env by running the interpreter only once, while allowing several scripts to use it at the same time.

the idea was bot the save the interpreter startup overhead and to save RAM.

Does something alike exists?

this question Python: Execute multiple Scripts simultaneously from same Interpreter doesn't address concurrency. at least the answers were about sequential running, but i need simultaneously :)

ideas?

Answer 1

Yes and no. Python itself uses a Global Interpreter Lock (GIL), which you can read a lot about, if you care to. To make a long story short, however, it ensures the interpreter is basically single-threaded. You can create (and run) more than one thread in your Python program, but when/if they use the Python interpreter, only one can do so at a time. If, however, you have threads running mostly code from something like SciPy or NumPy (which is native code that doesn't get interpreted) then you can run several concurrently.

Most operating systems, however, have a Copy On Write mechanism for process memory pages, which means that (as long as the code isn't modified) most of the code used by the interpreter will be shared without any extra work on your part (or the interpreter's) at all. IOW, when you run two or more copies of the interpreter, the second and subsequent will share most of the memory (at least for executable code) with the first, so resource usage will not rise (anywhere close to) linearly as you run more instances. Startup time will also be substantially reduced -- the OS has to create a new page table mapping the memory pages to the new process, but does not need to reread those pages from disk or anything like that.

Answer 2

Python supports threading via the thread and threading modules (one is lowlevel, the other one highlevel).