Why is this C++ code execution so slow compared to java?

Question

I recently wrote a computation-intensive algorithm in Java, and then translated it to C++. To my surprise the C++ executed considerably slower. I have now written a much shorter Java test program, and a corresponding C++ program - see below. My original code featured a lot of array access, as does the test code. The C++ takes 5.5 times longer to execute (see comment at end of each program).

Conclusions after 1^st 21 comments below ...

Test code:

1. g++ -o ... Java 5.5 times faster
2. g++ -O3 -o ... Java 2.9 times faster
3. g++ -fprofile-generate -march=native -O3 -o ... (run, then g++ -fprofile-use etc) Java 1.07 times faster.

My original project (much more complex than test code):

1. Java 1.8 times faster
2. C++ 1.9 times faster
3. C++ 2 times faster

Software environment:
    Ubuntu 16.04 (64 bit).
    Netbeans 8.2 / jdk 8u121 (java code executed inside netbeans)
    g++ (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
    Compilation: g++ -o cpp_test cpp_test.cpp

Java code:

public class JavaTest {
    public static void main(String[] args) {
        final int ARRAY_LENGTH = 100;
        final int FINISH_TRIGGER = 100000000;
        int[] intArray = new int[ARRAY_LENGTH];
        for (int i = 0; i < ARRAY_LENGTH; i++) intArray[i] = 1;
        int i = 0;
        boolean finished = false;
        long loopCount = 0;
        System.out.println("Start");
        long startTime = System.nanoTime();
        while (!finished) {
            loopCount++;
            intArray[i]++;
            if (intArray[i] >= FINISH_TRIGGER) finished = true;
            else if (i <(ARRAY_LENGTH - 1)) i++;
            else i = 0;
        }
        System.out.println("Finish: " + loopCount + " loops; " +
            ((System.nanoTime() - startTime)/1e9) + " secs");
        // 5 executions in range 5.98 - 6.17 secs (each 9999999801 loops)
    }
}

C++ code:

//cpp_test.cpp:
#include <iostream>
#include <sys/time.h>
int main() {
    const int ARRAY_LENGTH = 100;
    const int FINISH_TRIGGER = 100000000;
    int *intArray = new int[ARRAY_LENGTH];
    for (int i = 0; i < ARRAY_LENGTH; i++) intArray[i] = 1;
    int i = 0;
    bool finished = false;
    long long loopCount = 0;
    std::cout << "Start\n";
    timespec ts;
    clock_gettime(CLOCK_REALTIME, &ts);
    long long startTime = (1000000000*ts.tv_sec) + ts.tv_nsec;
    while (!finished) {
        loopCount++;
        intArray[i]++;
        if (intArray[i] >= FINISH_TRIGGER) finished = true;
        else if (i < (ARRAY_LENGTH - 1)) i++;
        else i = 0;
    }
    clock_gettime(CLOCK_REALTIME, &ts);
    double elapsedTime =
        ((1000000000*ts.tv_sec) + ts.tv_nsec - startTime)/1e9;
    std::cout << "Finish: " << loopCount << " loops; ";
    std::cout << elapsedTime << " secs\n";
    // 5 executions in range 33.07 - 33.45 secs (each 9999999801 loops)
}

Answer 1

The only time I could get the C++ program to outperform Java was when using profiling information. This shows that there's something in the runtime information (that Java gets by default) that allows for faster execution.

There's not much going on in your program apart from a non-trivial if statement. That is, without analysing the entire program, it's hard to predict which branch is most likely. This leads me to believe that this is a branch misprediction issue. Modern CPUs do instruction pipelining which allows for higher CPU throughput. However, this requires a prediction of what the next instructions to execute are. If the guess is wrong, the instruction pipeline must be cleared out, and the correct instructions loaded in (which takes time).

At compile time, the compiler doesn't have enough information to predict which branch is most likely. CPUs do a bit of branch prediction as well, but this is generally along the lines of loops loop and ifs if (rather than else).

Java, however, has the advantage of being able to use information at runtime as well as compile time. This allows Java to identify the middle branch as the one that occurs most frequently and so have this branch predicted for the pipeline.