What are the downsides of JIT compilation?

Question

Just In Time Compilation has a number of theoretical advantages: it has more information about the machine, the state of flags, and how the code is used. It allows you to avoid time-consuming compilation before running, speeding up the development cycle.

Simplistically, JIT compilation seems superior as an approach; there is some overhead, but if Sufficiently Smart it can speed up your code enough to offset that.

However, I don't think that's the whole story. What are the theoretical and practical downsides? Yes, "slower startup time" is often mentioned, as is "increased memory consumption", but I wonder if there is more than that.

For example, does tracing and JIT compilation trash the cpu cache? If your program is large and doesn't really have many particularly hot paths, is there a risk of spending more time tracing and JIT-ting than would be worthwhile?

It seems likely that someone has written a paper on this or on solving problems inherent in JIT. If someone has measurements, all the better.

Edit: I'm talking about Just In Time compilation in comparison to ahead of time compilation (potentially with feedback directed optimization), not in comparison to interpretation.

Answer 1

For example, does tracing and JIT compilation trash the cpu cache? If your program is large and doesn't really have many particularly hot paths, is there a risk of spending more time tracing and JIT-ting than would be worthwhile?

That's plausible. But the whole optimization game is about trading off various factors to achieve the best result in the average case. Most applications do have relatively hot and cold paths, even if the hot paths are all in the standard class libraries.

Besides what you are effectively saying is that this hypothetical application is not worth JIT compiling at all. In that case, the "fix" would be to run it with JIT compilation turned off.

It seems likely that someone has written a paper on this or on solving problems inherent in JIT.

You'd have thought so.

But the flip-side is that the folks who build and maintain the JIT compilers in the Oracle, IBM, etc JVMs may be restricted from telling the world about their ideas and results ... for commercial reasons. (Publishing the source code is one thing, but explaining why they chose a particular strategy is something else.) There's also the issue of motivation.

Answer 2

If the JIT is working properly, there is no real disadvantage. Having said that, it's taken Sun a long time to stabilize Hotspot because it's extremely complex. With regards to benchmark data, you can always run the following experiment:

Run SPECjbb, SPECjvm, or your own benchmark and modify the command line that executes java to include:

-Xint

This will exclude any runtime compilation from occuring.