Why is it hard to beat AOT compiler with a JIT compiler (in terms of app. performance)?

Question

I was thinking that JIT compilers will eventually beat AOT compilers in terms of the performance of the compiled code, due to the inherent advantage of JIT (can use information available only at runtime). One argument is that AOT compilers can spend more time compiling code, but a server VM could spend a lot of time, too.

I do understand that JIT does seem to beat AOT compilers in some cases, but they still seem to lag behind in most cases.

So my question is, what are the specific, tough problems that is preventing JIT compilers to beat AOT compilers?

EDIT:
Some common arguments:

• AOT compilers can spend more time doing advanced optimizations -> If you are running a server VM for days, you can spend the same amount of time, if not longer.
• Byte code interpretation has cost -> Most JIT compilers cache native machine instructions anyways these days.

Yet another edit:
For a specific example, see this article: Improving Swing Performance: JIT vs AOT Compilation. From what I can gather from this article, the authors are basically saying that when there are no hotspots, the advantage of having runtime information decreases and thus AOT without the overhead of JIT, wins. But by 40%?? That doesn't seem to make a lot of sense. Is it simply that the JIT compiler that was compared wasn't tuned for this situation? Or is it something more fundamental?

Answer 1

There's a definite trade-off between JIT and AOT (ahead-of-time) compilation.

As you stated, JIT has access to run-time information that can aid in optimization. This includes data about the machine it's executing on, enabling platform-specific native optimization. However, JIT also has the overhead of translating byte-code to native instructions.

This overhead often becomes apparent in applications where a fast start-up or near real-time responses are necessary. JIT is also not as effective if the machine does not have sufficient resources for advanced optimization, or if the nature of the code is such that it cannot be "aggressively optimized."

For example, taken from the article you linked:

... what should we improve in the absence of clear performance bottlenecks? As you may have guessed, the same problem exists for profile-guided JIT compilers. Instead of a few hot spots to be aggressively optimized, there are plenty of "warm spots" that are left intact.

AOT compilers can also spend as much time optimizing as they like, whereas JIT compilation is bound by time requirements (to maintain responsiveness) and the resources of the client machine. For this reason AOT compilers can perform complex optimization that would be too costly during JIT.

Also see this SO question: JIT compiler vs offline compilers