what is the difference between warmup attribute in Fork and Warmup annotation in jmh?

Question

I'm learning JMH benchmarking using this tutorial.

I noticed that there are 2 warmup related stuff for function benchMurmur3_128 in here.

so, I have question regarding the difference between warmup attribute in Fork annotation and Warmup annotation with iterations attribute?

Answer 1

With a JMH benchmark you run one or more forks sequentially, and one or more iterations of your benchmark code within each fork. There are two forms of warmup associated with this:

• At the fork level the warmups parameter to @Fork specifies how many warmup forks to run before running the benchmarked forks. Warmup forks are ignored when creating the benchmark results.
• The @Warmup annotation lets you specify warmup characteristics within a fork, including how many warmup iterations to run. Warmup iterations are ignored when creating the benchmark results.

For example:

• @Fork(value = 3, warmups = 2) means that 5 forks will be run sequentially. The first two will be warmup runs which will be ignored, and the final 3 will be used for benchmarking.
• @Warmup(iterations = 5, time = 55, timeUnit = TimeUnit.MILLISECONDS) means that there will be 5 warmup iterations within each fork. The timings from these runs will be ignored when producing the benchmark results.
• @Measurement(iterations = 4, time = 44, timeUnit = TimeUnit.MILLISECONDS) means that your benchmark iterations will be run 4 times (after the 5 warmup iterations).

So the overall impact of the warmup settings shown above is that:

• Only the final three of the five forks will be used for the benchmark results.
• Only the final four iterations within each non-warmup fork will be used for the benchmark results.

That is why the JMH output below (which was run using those annotations against the benchmarked method) shows Cnt 12 at the end of the run: 3 forks x 4 iterations = 12.