Use JUnit5 Tags for Spek

Question

I am trying to distinguish my tests into Unit- and Integration tests. My idea was to use the new JUnit5 Annotation @Tag("unit") which works nicely for my JUnit tests, but I cannot get it to work with Spek.

What I currently have is my class:

data class MyObject(val value: Int)

My tests:

@Tag("unit")
object MyObjectTest {

    @Test
    fun checkEquality() {
        val o1 = MyObject(1)
        assertEquals(o1, o1)
    }
}

With my build.gradle having:

task utest(type: Test) {
    outputs.upToDateWhen { false }
    useJUnitPlatform {
        includeEngines 'junit-jupiter', 'junit-vintage', 'spek'
        includeTags 'unit'
        excludeTags 'performance', 'integration', 'functional'
    }


    testLogging {
        events "passed", "skipped", "failed"
    }
}

When I execute utest, this works. However when doing the same with Spek:

@Tag("unit")
object MyObjectSpek : Spek({

   given("an Object") {
       val o1 = MyObject(1)

       it("should be equal to itself") {
           assertEquals(o1, o1)
       }
   }
})

What happens is if I run the gradle task utest it only executes the methods from MyObjectTest and does not execute the tests for MyObjectSpek

Any ideas on how to integrate Spek with JUnit5 Tags or another idea to seperate unit tests and integration tests?

Answer 1

today I ran exactly into the same problem. I had to separate tests into 3 sections: Unit, Service (testing REST API) and Integration (WebDriver).

Disclamer: this guide is applicable for any testing framework, not only to Spek. Gradle 4.6 or newer is required to run this.

Separate test source set into source sets

In my example they would be:

• src/test — for Unit tests (you already have it)
• src/serviceTest — for Service tests
• src/integrationTest — for Integration tests

all these sets should have standard source set structure. Create these folders inside your project and move your packages to corresponding source sets.

When it is done add to build.gradle before dependency section following lines:

sourceSets {
    integrationTest {
        compileClasspath += main.output
        runtimeClasspath += main.output
    }
    serviceTest {
        compileClasspath += main.output
        runtimeClasspath += main.output
    }
}

configurations {
    integrationTestCompile.extendsFrom testCompile
    integrationTestRuntime.extendsFrom testRuntime

    serviceTestCompile.extendsFrom testCompile
    serviceTestRuntime.extendsFrom testRuntime
}

After you do this your IDE (I suppose you use Idea) should reindex build.gradle and recognize source sets. You may have errors in your new source sets because they do not see each other sources. That's correct, because these source sets intended to run independently, and should not be a problem.

Separate dependencies to appropriate configurations (Optional)

By default serviceTest and integrationTest inherit all test dependencies, but if you need to move something specific to certain configuration out of common scope, you can do this here.

In my case WebDriver is quite heavy and I do not need it anywhere except integration testing.

dependencies {
    // available for all scopes
    testCompile "org.jetbrains.spek:spek-api:$spekVersion"
    testRuntime "org.jetbrains.spek:spek-junit-platform-engine:$spekVersion"
    testCompile "org.junit.platform:junit-platform-launcher:$junitPlatformVersion"

    // compiles only for integrationTest
    integrationTestCompile "org.seleniumhq.selenium:selenium-java:3.11.0"
    integrationTestCompile "org.seleniumhq.selenium.fluent:fluent-selenium:1.19"
}

Setup execution order

We will need to add gradle task of Test type and setup it. You can have different settings for different test tasks.

task serviceTest(type: Test) {
    // Runs tests from src/serviceTest
    testClassesDirs = sourceSets.serviceTest.output.classesDirs
    classpath = sourceSets.serviceTest.runtimeClasspath
}
// Setup serviceTest task 
serviceTest {
    // Uncomment this if you need to skip tests from the set after first failure. Since Gradle 4.6
    //failFast = true

    // Enable some logging
    testLogging {
        events "PASSED", "FAILED", "SKIPPED"
    }

    // Enable JUnit5 tests
    useJUnitPlatform {
    }
}

Do the same for integrationTest.

Finally, setup the dependencies and order of execution:

// Make service tests run during gradle check
check.dependsOn serviceTest
check.dependsOn integrationTest

// Make service tests run after unit tests
serviceTest.mustRunAfter test
// Make integration tests run after service tests
integrationTest.mustRunAfter serviceTest

Conclusion

You will get:

1. Chain of Unit -> Service -> Integration test suites running in strict order;
2. If you will get a test failure (regardless of failFast option) in one test suite, the remaining won't run and waste resources;
3. Ability to run from console each suite separately within execution of gradle <task>.

Additional resources:

• Here is the sample build.gradle that implements this setup with Spek
• Release notes of gradle 4.6 version introducing a lot of stuff related to testing.