Programmatically launch and interact with python virtual machine

Question

Does anyone know how to launch a new python virtual machine from inside a python script, and then interact with it to execute code in a completely separate object space? In addition to code execution, I'd like to be able to access the objects and namespace on this virtual machine, look at exception information, etc.

I'm looking for something similar to python's InteractiveInterpreter (in the code module), but as far as I've been able to see, even if you provide a separate namespace for the interpreter to run in (through the locals parameter), it still shares the same object space with the script that launched it. For instance, if I change an attribute of the sys module from inside InteractiveInterpreter, the change takes affect in the script as well. I want to completely isolate the two, just like if I was running two different instances of the python interpreter to run two different scripts on the same machine.

I know I can use subprocess to actually launch python in a separate process, but I haven't found any good way to interact with it the way I want. I imagine I could probably invoke it with '-i' and push code to it through it's stdin stream, but I don't think I can get access to its objects at all.

Answer 1

If you start a new Python instance with subprocess, afterwards you can communicate through sockets: to avoid doing the low level stuff you could have a look at the twisted framework, also have a look at Pyro http://pythonhosted.org/Pyro4/. Personally I find Pyro a bit easier to get started.

Answer 2

It may depend on Python implementation such as Pypy, Jython. In CPython, you have to use a separate process if you want an independent interpreter otherwise at the very least GIL is shared.

multiprocessing, concurrent.futures modules allow you to run arbitrary Python code in separate processes and to communicate with the parent easily.