Just-in-time compilation - when does it actually take place in Java?

Question

Recently I took part in a discussion over the efficiency of Java. As I listened, plenty of arguments against Java were that interpreting is "extremely time consuming" and because of it even simple Java programs run way slower than similar one's compiled directly to machine code.

An answer to that was that Java code is often compiled directly to machine code, if only JVM calculates it makes program faster than if it was interpreted in a standard way.

My questions are: when does JVM actually "decide" to perform just-in-time compilation? What are the criteria that make JIT more efficient than standard bytecode interpreting? I mean, the compilation takes some time itself, and as far as I understand, it is all supposed to happen when the program is already running?

Answer 1

This varies widely depending on your JVM and its settings. Wikipedia:

For example, Sun's Java Virtual Machine has two major modes—client and server. In client mode, minimal compilation and optimization is performed, to reduce startup time. In server mode, extensive compilation and optimization is performed, to maximize performance once the application is running by sacrificing startup time. Other Java just-in-time compilers have used a runtime measurement of the number of times a method has executed combined with the bytecode size of a method as a heuristic to decide when to compile.[4] Still another uses the number of times executed combined with the detection of loops.[5]

A rough approximation for a vanilla HotSpot JVM in -server mode is that JIT occurs when the JVM notices that a certain method has been called a lot, usually more than some specific number of times. (This is why the JVM is called "HotSpot" -- because it identifies and optimizes "hot spots" in your code.) At this point, the JVM knows a few things:

• It knows that this method is worth spending the time to optimize, because, well, it gets called a lot.
• It knows a lot about the real-world characteristics of this function:
  • if one branch of an if statement is much more common than another, so it can improve branch prediction
  • if e.g. a List passed to this method is usually an ArrayList, so it can make optimizations and inlinings for the specific case of an ArrayList.

Note that if you compiled in advance to machine code, you wouldn't have a lot of this information to optimize with -- the compiler doesn't know in advance which paths tend to be more common than others. But if you wait to do your optimization until you have real-world data, you can make better optimization decisions.

Some more details on what the JIT does are here.