Java - Vector vs ArrayList performance - test

Question

Everybody's saying that one should use vector because of the perfomance (cause Vector synchronizes after every operation and stuff). I've written a simple test:

import java.util.ArrayList;
import java.util.Date;
import java.util.Vector;

public class ComparePerformance {

    public static void main(String[] args) {
        ArrayList<Integer> list = new ArrayList<Integer>();
        Vector<Integer> vector = new Vector<Integer>();

        int size = 10000000;
        int listSum = 0;
        int vectorSum = 0;

        long startList = new Date().getTime();
        for (int i = 0; i < size; i++) {
            list.add(new Integer(1));
        }
        for (Integer integer : list) {
            listSum += integer;
        }
        long endList = new Date().getTime();
        System.out.println("List time: " + (endList - startList));

        long startVector = new Date().getTime();
        for (int i = 0; i < size; i++) {
            vector.add(new Integer(1));
        }
        for (Integer integer : list) {
            vectorSum += integer;
        }
        long endVector = new Date().getTime();
        System.out.println("Vector time: " + (endVector - startVector));
    }
}

The results are as follows:

List time: 4360
Vector time: 4103

Based on this it seems that Vector perfomance at iterating over and reading is slightly better. Maybe this is a dumb queston or I've made wrong assumptions - can somebody please explan this?

Answer 1

You have written a naïve microbenchmark. Microbenchmarking on the JVM is very tricky business and it is not even easy to enumerate all the pitfalls, but here are some classic ones:

1. you must warm up the code;
2. you must control for garbage collection pauses;
3. System.currentTimeMillis is imprecise, but you don't seem to be aware of even this method (your new Date().getTime() is equivalent, but slower).

If you want to do this properly, then check out Oracle's jmh tool or Google's Caliper.

My Test Results

Since I was kind of interested to see these numbers myself, here is the output of jmh. First, the test code:

public class Benchmark1
{
  static Integer[] ints = new Integer[0];
  static {
    final List<Integer> list = new ArrayList(asList(1,2,3,4,5,6,7,8,9,10));
    for (int i = 0; i < 5; i++) list.addAll(list);
    ints = list.toArray(ints);
  }
  static List<Integer> intList = Arrays.asList(ints);
  static Vector<Integer> vec = new Vector<Integer>(intList);
  static List<Integer> list = new ArrayList<Integer>(intList);

  @GenerateMicroBenchmark
  public Vector<Integer> testVectorAdd() {
    final Vector<Integer> v = new Vector<Integer>();
    for (Integer i : ints) v.add(i);
    return v;
  }
  @GenerateMicroBenchmark
  public long testVectorTraverse() {
    long sum = (long)Math.random()*10;
    for (int i = 0; i < vec.size(); i++) sum += vec.get(i);
    return sum;
  }
  @GenerateMicroBenchmark
  public List<Integer> testArrayListAdd() {
    final List<Integer> l = new ArrayList<Integer>();
    for (Integer i : ints) l.add(i);
    return l;
  }
  @GenerateMicroBenchmark
  public long testArrayListTraverse() {
    long sum = (long)Math.random()*10;
    for (int i = 0; i < list.size(); i++) sum += list.get(i);
    return sum;
  }
}

And the results:

testArrayListAdd          234.896  ops/msec
testVectorAdd             274.886  ops/msec
testArrayListTraverse    1718.711  ops/msec
testVectorTraverse         34.843  ops/msec

Note the following:

• in the ...add methods I am creating a new, local collection. The JIT compiler uses this fact and elides the locking on Vector methods—hence almost equal performance;
• in the ...traverse methods I am reading from a global collection; the locks cannot be elided and this is where the true performance penalty of Vector shows up.

The main takeaway from this should be: the performance model on the JVM is highly complex, sometimes even erratic. Extrapolating from microbenchmarks, even when they are done with all due care, can lead to dangerously wrong predictions about production system performance.