Why is there such a performance difference on Raspberry pi between Open and Oracle JDK?

Question

On my Raspberry I did some Performance Tests like CaffeineMark and SciMark with both JVMs. There is a huge performance difference between them even though I have heard that the differences are very small. I also tried calculating with floating numbers and the Oracle JDK got a better Score even though both should support the hard float abi.

I use Linux raspberrypi 3.18.11-v7+ as OS.

OpenJDK:    
java version "1.7.0_79"  
OpenJDK Runtime Environment (IcedTea 2.5.5) (7u79-2.5.5-1~deb7u1+rpi1)  
OpenJDK Zero VM (build 24.79-b02, mixed mode)  

OracleJDK:  
java version "1.7.0_40"  
Java(TM) SE Runtime Environment (build 1.7.0_40-b43)  
Java HotSpot(TM) Client VM (build 24.0-b56, mixed mode)  

SciMark results:

                 OpenJDK            OracleJDK
Composite Score 14.280735577363213  || 32.24948180361924   
FFT (1024)      9.482866845055302   || 26.447121360843663  
SOR (100x100)   27.14938943220646   || 59.68022533004399  
Monte Carlo     3.6298604956147384  || 10.561671865446971  
Sparse matmult  15.603809523809524  || 26.64931580928407  
LU (100x100)    15.53775159013005   || 37.90907465247749 

I used a program that counts a float in 0.1 steps to 600000. I tried to test the performance of the JVM on handling floats.

OpenJDK: 257ms
OracleJDK: 151ms

0.1 steps to 1200000:

OpenJDK: 457ms
OracleJDK: 263ms

public class Testing {

    /**
     * @param args
     */
    public static long Test()
    {
        float counter=0.0f;
        long startTime = System.currentTimeMillis();
        while (counter <= 1_200_000.0f)
        {
            counter += 0.1f;
        }
        return System.currentTimeMillis() - startTime;
    }

    public static void main(String[] args){
        System.out.println(Test());
    }

}

I tried the enhancements mentioned from SlipperySeal and put the test in the loop. I also tried to use the c2 compiler but the results were not different.

Answer 1

OpenJDK Zero VM is an interpreter only JVM. On the one hand, it's easier to port because it has no architecture specific assembly code but, on the other hand, it's not performant because it has no architecture specific assembly code.

OracleJDK takes advantage of the platform's floating point ABI (Soft Float on RP1 and Hard Float on RP2). I can imagine it has quite an amount of assembly code, specific to the ARM architecture, and that is why it scores better.

A JIT compiler named Shark, based on LLVM, was introduced in OpenJDK Zero VM early on. I'm not sure if your system's OpenJDK was build with Shark but it probably was. It provides a compromise between not having assembly code and still running efficient native code. If Shark is not enabled then building IcedTea with Shark enabled will improve performance. If Shark is enabled then that is the reason why OpenJDK does not suck as much.

Answer 2

An update for April 2018.

I run a Java Whetstone off-line benchmark on Raspberry Pi 3 with Raspbian Stretch. The source code can be found here in file Raspberry_Pi_Benchmarks/java/source code/whetstone-off-line/whetstc.java.

The conclusion is that openjdk-9-jre is faster than openjdk-8-jre (tested version 1.8.0_162), which is in turn faster than oracle-java8-jdk (tested version 1.8.0_65). Note also that I used an older kernel with oracle-java8-jdk.

openjdk-9-jdk from Raspbian repos:

$ java -version
openjdk version "9-Raspbian"
OpenJDK Runtime Environment (build 9-Raspbian+0-9b181-4bpo9rpt1)
OpenJDK Server VM (build 9-Raspbian+0-9b181-4bpo9rpt1, mixed mode)

$ java whetstc 
   Whetstone Benchmark Java Version, apr. 26 2018, 23:15:40

                                                     1 Pass
Test                  Result       MFLOPS     MOPS  millisecs

N1 floating point  -1,124750137    333,22             0,0576
N2 floating point  -1,131330490    293,19             0,4584
N3 if then else     1,000000000             185,95    0,5566
N4 fixed point     12,000000000             412,95    0,7628
N5 sin,cos etc.     0,499110132              22,40    3,7140
N6 floating point   0,999999821    212,70             2,5360
N7 assignments      3,000000000             105,66    1,7490
N8 exp,sqrt etc.    0,825148463              16,70    2,2280

MWIPS                              829,02            12,0624

Operating System    Linux, Arch. arm, Version 4.14.34-v7+
Java Vendor         Oracle Corporation, Version  9-Raspbian
CPU null

openjdk-8-jdk from Raspbian repos:

$ java -version
openjdk version "1.8.0_162"
OpenJDK Runtime Environment (build 1.8.0_162-8u162-b12-1~deb9u1-b12)
OpenJDK Client VM (build 25.162-b12, mixed mode)

$ java whetstc
   Whetstone Benchmark Java Version, Apr 27 2018, 13:13:26

                                                     1 Pass
Test                  Result       MFLOPS     MOPS  millisecs

N1 floating point  -1.124750137    181.82             0.1056
N2 floating point  -1.131330490    175.92             0.7640
N3 if then else     1.000000000              88.61    1.1680
N4 fixed point     12.000000000             389.85    0.8080
N5 sin,cos etc.     0.499110132               9.35    8.8980
N6 floating point   0.999999821     76.27             7.0720
N7 assignments      3.000000000             275.82    0.6700
N8 exp,sqrt etc.    0.825148463               7.15    5.2060

MWIPS                              405.00            24.6916

Operating System    Linux, Arch. arm, Version 4.14.34-v7+
Java Vendor         Oracle Corporation, Version  1.8.0_162
CPU null

oracle-java8-jdk from Raspbian repos:

$ java -version
java version "1.8.0_65"
Java(TM) SE Runtime Environment (build 1.8.0_65-b17)
Java HotSpot(TM) Client VM (build 25.65-b01, mixed mode)`

$ java whetstc
   Whetstone Benchmark Java Version, Nov 15 2017, 11:16:37

                                                     1 Pass
Test                  Result       MFLOPS     MOPS  millisecs

N1 floating point  -1.124750137     91.52             0.2098
N2 floating point  -1.131330490     89.01             1.5100
N3 if then else     1.000000000              44.27    2.3380
N4 fixed point     12.000000000             229.76    1.3710
N5 sin,cos etc.     0.499110103               3.01   27.6400
N6 floating point   0.999999821     44.95            12.0000
N7 assignments      3.000000000             137.09    1.3480
N8 exp,sqrt etc.    0.751108646               0.58   63.9100

MWIPS                               90.64           110.3268

Operating System    Linux, Arch. arm, Version 4.9.35-v7+
Java Vendor         Oracle Corporation, Version  1.8.0_65
CPU null