java immutable class much slower

Question

I was in need of some Complex math library, so I hesitated between libraries that use immutable Complex and libraries that use mutable Complex. Obviously, I want computations to run reasonably fast (unless it kills readability etc.).

So I created simple test of speed mutable vs immutable:

final class MutableInt {
    private int value;

    public int getValue() {
        return value;
    }

    public void setValue(int value) {
        this.value = value;
    }

    public MutableInt() {
        this(0);
    }

    public MutableInt(int value) {
        this.value = value;
    }   
}

final class ImmutableInt {
    private final int value;

    public ImmutableInt(int value) {
        this.value = value;
    }

    public int getValue() {
        return value;
    }
}

public class TestImmutableSpeed {

    static long testMutable(final int arrLen) {
        MutableInt[] arrMutable = new MutableInt[arrLen];
        for (int i = 0; i < arrMutable.length; ++i) {
            arrMutable[i] = new MutableInt(i);
            for (int j = 0; j < arrMutable.length; ++j) {
                arrMutable[i].setValue(arrMutable[i].getValue() + j);
            }
        }
        long sumMutable = 0;
        for (MutableInt item : arrMutable) {
            sumMutable += item.getValue();
        }
        return sumMutable;
    }

    static long testImmutable(final int arrLen) {
        ImmutableInt[] arrImmutable = new ImmutableInt[arrLen];
        for (int i = 0; i < arrImmutable.length; ++i) {
            arrImmutable[i] = new ImmutableInt(i);
            for (int j = 0; j < arrImmutable.length; ++j) {
                arrImmutable[i] = new ImmutableInt(arrImmutable[i].getValue() + j);
            }
        }
        long sumImmutable = 0;
        for (ImmutableInt item : arrImmutable) {
            sumImmutable += item.getValue();
        }
        return sumImmutable;
    }

    public static void main(String[] args) {
        final int arrLen = 1<<14;

        long tmStart = System.nanoTime();
        System.out.println("sum = " + testMutable(arrLen));
        long tmMid = System.nanoTime();
        System.out.println("sum = " + testImmutable(arrLen));
        long tmEnd = System.nanoTime();

        System.out.println("speed comparison mutable vs immutable:");
        System.out.println("mutable   " + (tmMid - tmStart)/1000000 + " ms");
        System.out.println("immutable " + (tmEnd - tmMid)/1000000 + " ms");
    }
}

You can adjust size of array if the test runs too slow/fast.

I run with: -server -Xms256m -XX:+AggressiveOpts And I get:

sum = 2199023247360
sum = 2199023247360
speed comparison mutable vs immutable:
mutable   102 ms
immutable 1506 ms

Question: Am I missing some optimization parameter, or is immutable version 15x slower?

If it is, why would anyone write math library with immutable class Complex in it? Is immutable just "fancy" but useless?

I know that immutable class is safer as hash map key or can't have race conditions, but that are special cases that can be handled without immutability everywhere.

Edit: I re-run this microbenchmark with caliper, as suggested by one answer, and it runs 12x slower, not 15x, still the same point. Changed code for Caliper benchmarking:

import com.google.caliper.Runner;
import com.google.caliper.SimpleBenchmark;



final class MutableInt {
    private int value;

    public int getValue() {
        return value;
    }

    public void setValue(int value) {
        this.value = value;
    }

    public MutableInt() {
        this(0);
    }

    public MutableInt(int value) {
        this.value = value;
    }   
}

final class ImmutableInt {
    private final int value;

    public ImmutableInt(int value) {
        this.value = value;
    }

    public int getValue() {
        return value;
    }
}


public class TestImmutableSpeed extends SimpleBenchmark {

    static long testMutable(final int arrLen) {
        MutableInt[] arrMutable = new MutableInt[arrLen];
        for (int i = 0; i 

Output of Caliper:

 0% Scenario{vm=java, trial=0, benchmark=Mutable, type=-server, minMemory=-Xms256m, optimizations=-XX:+AggressiveOpts} 91614044.60 ns; ?=250338.20 ns @ 3 trials
50% Scenario{vm=java, trial=0, benchmark=Immutable, type=-server, minMemory=-Xms256m, optimizations=-XX:+AggressiveOpts} 1108057922.00 ns; ?=3920760.98 ns @ 3 trials

benchmark     ms linear runtime
  Mutable   91.6 ==
Immutable 1108.1 ==============================

Note that without the optimization parameters for JVM output of Caliper is:

 0% Scenario{vm=java, trial=0, benchmark=Mutable} 516562214.00 ns; ?=623120.57 ns @ 3 trials
50% Scenario{vm=java, trial=0, benchmark=Immutable} 1706758503.00 ns; ?=5842389.60 ns @ 3 trials

benchmark   ms linear runtime
  Mutable  517 =========
Immutable 1707 ==============================

So bad parameters make both version slow, but ratio is less terrible (but not important, still).

Answer 1

This is fascinating. Well, first of all, that is not a fair test; you aren't warming up the JVM when you do it that way. Benchmarking is typically very difficult to do. I refactored your code to use Google Caliper, and got similar, yet different results; the immutable class was only 3 times slower. Not sure why yet. Anyway here's the work so far:

TestImmutableSpeed.java

import com.google.caliper.Runner;
import com.google.caliper.SimpleBenchmark;

public class TestImmutableSpeed {
    static final class MutableInt {
        private int value;

        public int getValue() {
            return value;
        }

        public void setValue(int value) {
            this.value = value;
        }

        public MutableInt() {
            this(0);
        }

        public MutableInt(int value) {
            this.value = value;
        }   
    }

    static final class ImmutableInt {
        private final int value;

        public ImmutableInt(int value) {
            this.value = value;
        }

        public int getValue() {
            return value;
        }
    }

    public static class TestBenchmark extends SimpleBenchmark {
        public void timeMutable(final int arrLen) {
            MutableInt[] arrMutable = new MutableInt[arrLen];
            for (int i = 0; i < arrMutable.length; ++i) {
                arrMutable[i] = new MutableInt(i);
                for (int j = 0; j < arrMutable.length; ++j) {
                    arrMutable[i].setValue(arrMutable[i].getValue() + j);
                }
            }
            long sumMutable = 0;
            for (MutableInt item : arrMutable) {
                sumMutable += item.getValue();
            }
            System.out.println(sumMutable);
        }

        public void timeImmutable(final int arrLen) {
            ImmutableInt[] arrImmutable = new ImmutableInt[arrLen];
            for (int i = 0; i < arrImmutable.length; ++i) {
                arrImmutable[i] = new ImmutableInt(i);
                for (int j = 0; j < arrImmutable.length; ++j) {
                    arrImmutable[i] = new ImmutableInt(arrImmutable[i].getValue() + j);
                }
            }
            long sumImmutable = 0;
            for (ImmutableInt item : arrImmutable) {
                sumImmutable += item.getValue();
            }
            System.out.println(sumImmutable);
        }
    }

    public static void main(String[] args) {
        Runner.main(TestBenchmark.class, new String[0]);
    }
}

Caliper Output

 0% Scenario{vm=java, trial=0, benchmark=Immutable} 78574.05 ns; σ=21336.61 ns @ 10 trials
 50% Scenario{vm=java, trial=0, benchmark=Mutable} 24956.94 ns; σ=7267.78 ns @ 10 trials

 benchmark   us linear runtime
 Immutable 78.6 ==============================
   Mutable 25.0 =========

 vm: java
 trial: 0

String update

So I was thinking about this more, and I decided to try changing the wrapped class from an int to an object, in this case a String. Changing the static classes to Strings, and loading the strings with Integer.valueOf(i).toString(), and instead of adding, appending them in a StringBuilder, I got these results:

 0% Scenario{vm=java, trial=0, benchmark=Immutable} 11034616.91 ns; σ=7006742.43 ns @ 10 trials
50% Scenario{vm=java, trial=0, benchmark=Mutable} 9494963.68 ns; σ=6201410.87 ns @ 10 trials

benchmark    ms linear runtime
Immutable 11.03 ==============================
  Mutable  9.49 =========================

vm: java
trial: 0

However, I think in this case the difference is dominated by all the array copying that would have to happen, and not the fact that it was using Strings.

Answer 2

Immutable values make clean programming in Java cleaner. You don't have to copy everywhere to avoid ending up with spooky action at a distance (by which I mean changing a value in one place inadvertently changes a value in another). Removing the copy speeds things up in places, but creating new instances slows things down in other areas.

(C++ is interesting in that it takes the opposite approach. You get copies at well defined points without having to write any code. Indeed you have to write code in order to remove the copying.)

If your concern is for performance, a mutable complex is no good either. Much better to have, say, a complex array class which uses a single double array hidden in the implementation, or just the double array raw.

Back in the nineties, Guy Steele mentioned the idea of adding value types to Java as part of making the language itself complete. Although that was a very limited proposal, similar structs C# later introduced, but neither copes with arguably the most obvious value class in Java, the string.