Are there any examples of multithreaded Tracing JIT compilers?

Question

Both the JVM and the .NET CLR include Just-In-Time compilers which support multiple user threads. However, I believe these are method-at-a-time JITs.

All of the tracing JITs I am aware of, for example LuaJIT and PyPy, are only single-threaded.

Are there any examples of tracing JITs which support multiple user threads? If not, are there any technical reasons why these do not exist?

Answer 1

Profiling (tracing) a running multi-threaded program is a lot harder, but also not impossible. The whole point of tracing is to make the runtime better than an optimizing compiler did the first time around. If the threads are interlinked, then the JIT that is going to modify the code needs to understand not just how the code is executed, but what the side effects are on other threads.

When thread one needs to access a big file in memory, does it create a level two cache flush that causes thread two to stall for a reason that is external to the code it is running. The JIT has to understand these interactions. Otherwise it might spend a lot of time trying to optimize thread two when improvements in thread two would come from realizing that thread one code is adversely effecting thread two and trying to eliminate the cache flush.

Are you considering trying to write your own tracing multi-threaded JIT? It could be done, but it is involved.

Answer 2

Your question is moot due to its wrong premise. The HotSpot optimizer of Oracle’s JVM/OpenJDK is not a “method-at-a-time JIT”. One of its fundamental technologies is the inlining capability, often called “aggressive inlining” as it does speculatively inline methods assumed to be most likely the target of a dynamic method dispatch, based on the profiling of the current execution (and other hints). This even includes the capability of de-optimizing, if the runtime behavior of the program changes and it doesn’t execute the optimized code path anymore.

The inlining is fundamental, as most other code optimizations develop their real potential only, when the code of methods is inlined into the caller’s, providing the necessary context.

So with the HotSpot JVM, you already have a multi-threaded optimizing environment utilizing known execution paths. This information doesn’t need to be gathered in the way described as “tracing”, though. Since this JVM is capable of creating a snapshot of a thread’s stack trace at any time, it can also peek the trace in defined time intervals, having more control over the profiling overhead than adding a recording feature to every method invocation. So, the JVM can limit the acquisition of traces to threads actually consuming significant CPU time and will intrinsically get an actual call chain, even if the involved methods are contained in multiple call chains of different threads.