JUnit 5 Parameterized Test - 'Cannot invoke non-static method' in @MethodSource with Kotlin

Question

Expect

Convert a standard JUnit 5 test into a parameterized test in order to iterate through a stream of test cases using the @ParamterizedTest and @MethodSource annotations in Kotlin.

Observe

The @MethodSource is unable to access the stream of data. This seems to be an issue with this annotation specifically, as @ValueSource(strings = ["SF", "NYC"]) iterates through the statically defined options as expected.

Error:

PreconditionViolationException: Cannot invoke non-static method {someMethodName} on a null target.

Implement

The parameterized test is set to pass in a stream of data classes similar to the setup outlined by Phillip Hauer in Data Classes for Parameterized Tests.

Code

build.gradle (:SomeProject)

dependencies {
    ...    
    classpath("de.mannodermaus.gradle.plugins:android-junit5:$junit5_version")
}

build.gradle (:someModule)

apply plugin: "de.mannodermaus.android-junit5"
android {
    ...
    compileOptions.targetCompatibility = JavaVersion.VERSION_1_8
    kotlinOptions.jvmTarget = "1.8"
}
dependencies {
    testImplementation "org.junit.jupiter:junit-jupiter-api:5.6.2"
    testRuntimeOnly "org.junit.jupiter:junit-jupiter-engine:5.6.2"
    testImplementation "org.junit.jupiter:junit-jupiter-params:5.6.2"
}

SomeTest.kt

class SomeTest {
    private val testDispatcher = TestCoroutineDispatcher()

    private fun someDataStates() = Stream.of(
        // Kotlin data class
        TestState("123"),
        TestState("345")
    )

    @ParameterizedTest
    @MethodSource("someDataStates")
    fun someTest(testState: TestState) = testDispatcher.runBlockingTest {
        // Test state here.
        ...
    }
}

Build environment

• Android Studio 4.0
• Build #AI-193.6911.18.40.6514223, built on May 20, 2020
• Runtime version: 1.8.0_242-release-1644-b3-6222593 x86_64
• VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
• macOS 10.15.5
• GC: ParNew, ConcurrentMarkSweep
• Memory: 1979M
• Cores: 16
• Registry: ide.new.welcome.screen.force=true
• Non-Bundled Plugins: cn.wjdghd.unique.plugin.id, com.android.tool.sizereduction.plugin, com.developerphil.adbidea, com.thoughtworks.gauge, mobi.hsz.idea.gitignore

Attempted solutions

1. Refactor test case data states to a top-level function.

TestCases.kt

fun someDataStates() = Stream.of(
    TestState("123"),
    TestState("345")
)

SomeTest.kt

private fun SomeDataStates() = someDataStates()

@ParameterizedTest
@MethodSource("SomeDataStates")
fun someTest(testState: TestState) = testDispatcher.runBlockingTest {
   // Test state here.
   ...

}

2. Refactor test case data states to a top-level function of type List instead of Stream.

fun someDataStates() = listOf(
    TestState("123"),
    TestState("345")
)

Full error log

org.junit.platform.commons.PreconditionViolationException: Cannot invoke non-static method [private final {someMethodName} on a null target.

at org.junit.platform.commons.util.Preconditions.condition(Preconditions.java:296) at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:682) at org.junit.jupiter.params.provider.MethodArgumentsProvider.lambda$provideArguments$1(MethodArgumentsProvider.java:46) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:272) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:272) at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:106) at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:41) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80) at java.util.ArrayList.forEach(ArrayList.java:1257) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80) at java.util.ArrayList.forEach(ArrayList.java:1257) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32) at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248) at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211) at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:132) at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:69) at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33) at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230) at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58) Suppressed: org.junit.platform.commons.PreconditionViolationException: Configuration error: You must configure at least one set of arguments for this @ParameterizedTest at org.junit.platform.commons.util.Preconditions.condition(Preconditions.java:281) at org.junit.jupiter.params.ParameterizedTestExtension.lambda$provideTestTemplateInvocationContexts$6(ParameterizedTestExtension.java:90) at java.util.stream.AbstractPipeline.close(AbstractPipeline.java:323) at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:279) ... 49 more

Process finished with exit code 255

Answer 1

I guess you're missing the information, to tell JUnit to instantiate your test class once, like this:

import org.junit.jupiter.api.TestInstance
import org.junit.jupiter.api.TestInstance.Lifecycle.PER_CLASS

@TestInstance(PER_CLASS) // <--- This one will do the trick
class SomeTest {

    private val testDispatcher = TestCoroutineDispatcher()

    @ParameterizedTest
    @MethodSource("someDataStates")
    fun someTest(testState: TestState) = testDispatcher.runBlockingTest {
        // Test state here.
        ...
    }

    private fun someDataStates() = listOf(
        TestState("123"),
        TestState("345")
    )
}

Answer 2

Using @TestInstance(PER_CLASS), you might have trouble for instance when trying to verify a function has been called X times with mockito or mockk. You'll have to sum all your calls.

In this case, use @JvmStatic :

class SomeTest {

    companion object{
        @JvmStatic
        private fun someDataStates() = listOf(
            TestState("123"),
            TestState("345")
        )
    }
    
    private val testDispatcher = TestCoroutineDispatcher()

    @ParameterizedTest
    @MethodSource("someDataStates")
    fun someTest(testState: TestState) = testDispatcher.runBlockingTest {
        // Test state here.
        ...
    }
}

Edit 27/08/2021: since latest version of kotlin (tested with 1.5.21), don't set the @JvmStatic method as private or it'll end again in a PreconditionViolationException

Answer 3

If for some reason you can't use the @TestInstance(PER_CLASS) annotation, you can use the @TestFactory annotation instead of the @ParameterizedTest annotation on your test method.

@TestFactory
Stream<DynamicTest> someTest() {
    return someDataStates().map(testState -> DynamicTest.dynamicTest(testState.getName(), () -> someTest0(testState)));
}

private void someTest0(TestState testState) {
    // Test state here.
}

private Stream<TestState> someDataStates() {
    return Stream.of(
            new TestState("123"),
            new TestState("345")
    );
}