Multi-threading in IronPython

Question

I have a "script class" in IronPython, and scripting in my app works by calling methods on its instance. I need to implement calling scripts from multiple threads. What is the correct way to do it?

I have multiple concerns:

1. Is ScriptScope thread-safe? Information is contradictory. ScriptScope's documentation says: "ScriptScope is not thread safe. Host should either lock when multiple threads could access the same module or should make a copy for each thread." However, IronRuby uses the same DLR and @JimmySchementi says that "ScriptRuntime, ScriptEngine, and ScriptScope are all thread safe, designed to be used between threads. Specifically, ScriptScope uses a thread-safe data-store, so ScriptScope can be shared between threads."

2. If I create multiple ScriptScopes, that would mean executing the same initialization script multiple times. Let's suppose that I run ten Python script files, import five assemblies and on the whole execute quite a bit of code to get the "script object" ready. Is there any way to avoid the time and memory cost of running a lot of the same code for each and every thread?

3. Is making the ScriptScope variable thread-static (that is, applying ThreadStaticAttribute) and executing initialization for every thread which is utilized by Task.Run the way to go? Or should I use a TaskScheduler with a limit on concurrency, because the cost of multiple scopes is high?

On the whole: how to correctly implement running the same script on different arguments in multiple threads? Scripts must be executing simultaneously and must not crash due to race conditions.

Answer 1

1.

If the documentation of ScriptScope says it is not thread safe, believe it, or at least act like you believe it. @JimmySchementi may have looked at the current implementation and worked out that it is currently thread safe, but this give no guarantees about how it will behave in the next patch of the class, let alone the next major release.

2.

Yes you will need to initialise each of your ScriptScopes. I would try and minimise the number of ScriptScopes you require, how to do this will depend on your set up. If the main purpose of the threads concerned is to host a ScriptScope then you should use a ThreadPool with each thread having one ThreadLocal<ScriptScope>. If these threads are doing other things as well as running scripts then you should have an Object pool storing the ScriptScopes and each thread can checkout the ScriptScopes, do the work, then release the ScriptScope.

3.

Prefer ThreadLocal over ThreadStatic if you go down this path.