Google Caliper: Trouble getting started (Java Benchmarking)

Question

I am trying to use Google Caliper to benchmark some simple code. I am using the examples from their websites. Here's what I've done so far:

1. Downloaded the Caliper JAR and added it to my Netbeans project
2. After having difficulties, I downloaded JUnit.jar and hamcrest.jar. Still not working.

Here's my code:

import com.google.caliper.Benchmark;

public class Benchmark1 extends Benchmark {

    public void timeNanoTime(int reps) {
      for (int i = 0; i < reps; i++) {
        System.nanoTime();
      }
    }

}

I am extending Benchmark because when I try to extend "SimpleBenchmark" like on their website it tells me it cannot find SimpleBenchmark. I then, in my main method, create a new Benchmark1() hoping something will happen. Nothing does. This is the code inside my main class.

Benchmark1 test = new Benchmark1();
test.timeNanoTime(10);

I know this is no doubt a simple error but I cannot, despite much Googling, figure out where I'm going wrong. The code compiles but does not run.

EDIT: I should say I'm running Netbeans on Windows 7 with Caliper 1.0

Answer 1

It's true; the documentation is woefully outdated and incomplete. I'm working on it. In the meantime, here's what will get your benchmark running.

Your main method should delegate to CaliperMain, not directly to the benchmark. Try

public static void main(String[] args) {
    CaliperMain.main(Benchmark1.class, args);
}

Windows will be a problem. Particularly, issue 215 will be the biggest blocker.

Answer 2

You could switch over to Perfidix http://perfidix.org/

Perfidix has eclipse Integration and can be used like JUnit.

Another option would be JUnitbenchmarks http://labs.carrotsearch.com/junit-benchmarks.html

It's a really great framework for Junit 4+. It can even build html charts to compare results.