Using jmh to benchmark code without creating separate maven project

Question

I am working on a Maven project and I wish to use jmh to benchmark my code. I want to organize my project so that it contains the source code, unit tests, and benchmarks. There seems to be a way in gradle to benchmark your code without creating a separate gradle project (see link). Is there a way to do this in Maven?

Answer 1

Short answer is yes.

I came across to this directory layout in my projects (but you definitely can change it)

+- src/
   +- main/java   - sources
   +- test/
      +- java        - test sources
      +- perf        - benchmarks

You need a couple of plugins to achieve that.

1. build-helper-maven-plugin to attach custom test sources location
   

<execution>
    <id>add-test-source</id>
    <phase>generate-test-sources</phase>
    <goals>
        <goal>add-test-source</goal>
    </goals>
    <configuration>
        <sources>
            <source>src/test/perf</source>
        </sources>
    </configuration>
</execution>

2. maven-compiler-plugin to run jmh-generator-annprocess annotation processor on test-compile phase
   

<execution>
    <goals>
        <goal>testCompile</goal>
    </goals>

    <configuration>
        <annotationProcessorPaths>
            <path>
                <groupId>org.openjdk.jmh</groupId>
                <artifactId>jmh-generator-annprocess</artifactId>
                <version>${jmh.version}</version>
            </path>
        </annotationProcessorPaths>
    </configuration>
</execution>

3. maven-assembly-plugin to create runnable jar with benchmarks
   

<execution>
    <id>make-assembly</id>
    <phase>package</phase>
    <goals>
        <goal>single</goal>
    </goals>
    <configuration>
        <attach>true</attach>
        <archive>
            <manifest>
                <mainClass>org.openjdk.jmh.Main</mainClass>
            </manifest>
        </archive>
    </configuration>
</execution>

<assembly>
    <id>perf-tests</id>
    <formats>
        <format>jar</format>
    </formats>
    <includeBaseDirectory>false</includeBaseDirectory>
    <dependencySets>
        <dependencySet>
            <outputDirectory>/</outputDirectory>
            <useProjectArtifact>true</useProjectArtifact>
            <unpack>true</unpack>
            <scope>test</scope>
        </dependencySet>
    </dependencySets>
    <fileSets>
        <fileSet>
            <directory>${project.build.directory}/test-classes</directory>
            <outputDirectory>/</outputDirectory>
            <includes>
                <include>**/*</include>
            </includes>
            <useDefaultExcludes>true</useDefaultExcludes>
        </fileSet>
    </fileSets>
</assembly>

After that you'll get an executable jar with benchmarks which could be run as usual

java -jar target/your-project-version-perf-tests.jar

You can see working example here.

NOTE

The only drawback of this solution is that all test classes and test dependencies will also be included in jar with benchmarks which definitely bloats it up. But you can avoid it by compiling benchmarks in separate (other than ${project.build.directory}/test-classes) directory.