Java 8 JIT thread seems like falling into infinite loop

Question

I wrote a server application in Java 8, and running it with java 1.8.0u25.

It works fine for the first several hours, but after it gets about 5k~10k requests, a thread of the VM process uses 100% of one of the CPUs.

So I tried jstack for the VM process to check what the problematic thread was, and it showed that the thread(thread id was 14303=0x37df) was "C2 CompilerThread0":

"C2 CompilerThread0" #6 daemon prio=9 os_prio=0 tid=0x00002aaabc12a000 nid=0x37df runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

And with jstack -m, the stack trace of the thread was as the following:

----------------- 14303 -----------------
0x00002b99b67693c3  _ZN16PhaseMacroExpand27process_users_of_allocationEP8CallNode + 0x2a3
0x00002b99b676ec3b  _ZN16PhaseMacroExpand23eliminate_allocate_nodeEP12AllocateNode + 0x1cb
0x00002b99b676ee65  _ZN16PhaseMacroExpand21eliminate_macro_nodesEv + 0x1a5
0x00002b99b6772769  _ZN16PhaseMacroExpand18expand_macro_nodesEv + 0x19
0x00002b99b640b01b  _ZN7Compile8OptimizeEv + 0xa6b
0x00002b99b640c53c  _ZN7CompileC1EP5ciEnvP10C2CompilerP8ciMethodibbb + 0x13bc
0x00002b99b635f9c8  _ZN10C2Compiler14compile_methodEP5ciEnvP8ciMethodi + 0x198
0x00002b99b6414c6a  _ZN13CompileBroker25invoke_compiler_on_methodEP11CompileTask + 0xc8a
0x00002b99b6417650  _ZN13CompileBroker20compiler_thread_loopEv + 0x620
0x00002b99b69a2e8f  _ZN10JavaThread17thread_main_innerEv + 0xdf
0x00002b99b69a2fbc  _ZN10JavaThread3runEv + 0x11c
0x00002b99b6860d48  _ZL10java_startP6Thread + 0x108

And every time I tried jstack -m, the stack trace of this thread was all the same, but the number beside the method(program counter?) at the top of the stack _ZN16PhaseMacroExpand27process_users_of_allocationEP8CallNode was 0x290, 0x2b1, 0x2a3, or 0x29f.

C2 CompilerThread0 looks like a thread doing JIT compilation, and the stack trace seems like it fell into an infinite loop or something.

I wonder if this could be a bug of JIT compiler of JVM. If it is, how can I specify which method of my application makes the JVM crazy and how can I solve(or work around) this problem? I tried -XX:+PrintCompilation option, but it did not help much because it didn't show which thread compiled which method. If it is not a problem of JVM, what could make this thing happen?

Answer 1

It indeed looks like a JIT compiler bug, presumably in allocation elimination optimization.
Try running with -XX:-EliminateAllocations JVM option.

You may also add -XX:+UnlockDiagnosticVMOptions -XX:+LogCompilation to produce detailed compilation log with a separate output file per compiler thread.