How to exec script without possibly changing the builtin sys module?

Question

I would like to exec() a script with a custom sys.path for that script, and letting it import modules without caching those modules into the builtin sys.modules.

I dont want to pollute the current interpreter sys.path and sys.modules, because different scripts are going to be executed simultaneously in a multithreaded environment.

So different scripts would be running simultaneously, with different paths each, importing different modules, but with possibly clashing names (thats why they need to run from different paths).

I thought that giving a different globals dictionary to each exec() would be enough, but the builtin sys module instance is actually shared among then.

Any ideas on how can I achieve that?

Note: I dont require a full sandbox solution - my scripts are from a trusted source, I just would like to run them all concurrently.

Answer 1

As @Rufflewind suggested in the comments, I ended up using the multiprocessing library. Each script is actually started on its own multiprocessing.Process instance. This library spawns another python interpreter process for each object, and thus, allows me to use different paths and modules, without interfering with the main script. The progress of the child process is shared with the main process by a multiprocessing.Queue object.

Because I am using a logging handler to write log records to a file, I have also used logging.handlers.QueueHandler to send these log records, generated on the child process, back to the main process. These records are then retrieved using a logging.handlers.QueueListener object.

Well, it turned out to be quite simple, indeed. =)