Menu
I am using the JMH benchmarking framework (http://openjdk.java.net/projects/code-tools/jmh/) to run benchmarks on my code. My understanding is that JMH forks the JVM multiple times during benchmarking in order to discard any profiles built up by the just-in-time (JIT) profiling performed by the JVM during execution.
I understand why this is useful in some cases such as the below (copied verbatim from http://java-performance.info/jmh/):
By default JHM forks a new java process for each trial (set of iterations). This is required to defend the test from previously collected “profiles” – information about other loaded classes and their execution information. For example, if you have 2 classes implementing the same interface and test the performance of both of them, then the first implementation (in order of testing) is likely to be faster than the second one (in the same JVM), because JIT replaces direct method calls to the first implementation with interface method calls after discovering the second implementation.
However, in the case where you are benchmarking the same code repeatedly is there any advantage to running say 10 forks of 20 iterations each instead of 1 fork with 200 iterations?
Many Thanks,
Danny
813
@Fork annotation, instructs how benchmark execution will happen the value parameter controls how many times the benchmark will be executed, and the warmup parameter controls how many times a benchmark will dry run before results are collected.
The JMH allows coders to design benchmarks that can measure either faster code sections or the much slower and longer functions. It works as a harness for microbenchmarks that are prepared in Java or other languages that are being targeted for use in the JVM environment.
JMH is popular for writing microbenchmarks, that is, benchmarks that stress a very specific piece of code. JMH also excels at concurrent benchmarks. That being said, JMH is a general-purpose benchmarking harness. It is useful for larger benchmarks, too.
There are two ways to run the JMH benchmark, uses Maven or run it via a JMH Runner class directly. 3.1 Maven, package it as a JAR and run it via org. openjdk.
Some people insist that another problem solved by using forks is run-to-run variance: http://hg.openjdk.java.net/code-tools/jmh/file/tip/jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_13_RunToRun.java
However, any serious engineer must be able to exercise sufficient control over the benchmarking environment to eliminate any difference between runs. It is disheartening to see people using forks to overcome their laziness or lack of understanding of how their benchmarks are actually executed.
193
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
