What does OpenJDK JMH "score error" exactly mean?

Question

I am using http://openjdk.java.net/projects/code-tools/jmh/ for benchmarking and i get a result like:

 Benchmark                         Mode   Samples        Score  Score error    Units o.a.f.c.j.b.TestClass.test1       avgt         5  2372870,600   210897,743    us/op o.a.f.c.j.b.TestClass.test2       avgt         5  2079931,850   394727,671    us/op o.a.f.c.j.b.TestClass.test3       avgt         5    26585,818    21105,739    us/op o.a.f.c.j.b.TestClass.test4       avgt         5    19113,230     8012,852    us/op o.a.f.c.j.b.TestClass.test5       avgt         5     2586,413     1949,487    us/op o.a.f.c.j.b.TestClass.test6       avgt         5     1942,963     1619,967    us/op o.a.f.c.j.b.TestClass.test7       avgt         5      233,902       73,861    us/op o.a.f.c.j.b.TestClass.test8       avgt         5      191,970      126,682    us/op 

What does the column "Score error" exactly mean and how to interpret it?

Answer 1

This is the margin of error for the score. In most cases, that is a half of confidence interval. Think about it as if there is a "±" sign between "Score" and "Score error". In fact, the human-readable log will show that:

Result: 1.986 ±(99.9%) 0.009 ops/ns [Average]   Statistics: (min, avg, max) = (1.984, 1.986, 1.990), stdev = 0.002   Confidence interval (99.9%): [1.977, 1.995]   # Run complete. Total time: 00:00:12  Benchmark                  Mode  Samples   Score  Score error   Units o.o.j.s.HelloWorld.hello  thrpt        5   1.986        0.009  ops/ns