scoverage: Combine Coverage from test and it:test

Question

I splitted my Unit- and Integration-Tests with a Filter:

  lazy val FunTest = config("it") extend Test

  def funTestFilter(name: String): Boolean = name endsWith "Spec"

  def unitTestFilter(name: String): Boolean = name endsWith "Test"

  ...
  testOptions in Test := Seq(Tests.Filter(unitTestFilter)),
  testOptions in FunTest := Seq(Tests.Filter(funTestFilter)),
  ...

So I can do something like that:

sbt clean coverage test dockerComposeUp it:test dockerComposeStop coverageReport

Sadly that kills all my Coverage, only the generated BuildInfo has a Coverage.

Using only sbt clean coverage test coverageReport or sbt clean coverage it:test coverageReport work as expected.

The whole project can be found here: https://github.com/pme123/play-binding-form

scoverage Version: 1.5.1

Answer 1

SBT supports incremental compilation, but Scoverage does not support it. Scoverage clears instrumentation information before compilation starts and starts instrumentation process from scratch every time. Compilation of a subset of all classes with Scoverage enabled will result in wrong coverage reports.

In this case sbt-buldinfo plugin is enabled in server module. It registers source generator, which is executed before every compilation and generates server/target/scala_2.12/src_managed/main/sbt-buildinfo/BuildInfo.scala file.

SBT BuildInfo plugin is smart enough to regenerate this file only when its content changes, but since BuildInfoOption.BuildTime is included in buildInfoOptions setting, this file is regeneraged before every compilation.

When it comes to compilation process, compiler finds one modified file (BuildInfo.scala) every time and starts incremental compilation of this one file. Scoverage clears its previous instrumentation information and saves only information about BuildInfo.scala file.

In case of execution like sbt clean coverage test dockerComposeUp it:test dockerComposeStop coverageReport the first compilation process is part of test task, and the second one it:test task. That's why there is no problem, when they are used separately.

Docker has nothing to do with our problem.

To fix the problem you have to prevent from BuildInfo.scala file regeneration on every compilation, at least when coverage is enabled. I did it by modifying project/Settings.scala file in this way:

  private lazy val buildInfoSettings = Seq(

    buildInfoKeys := Seq[BuildInfoKey](name, version, scalaVersion, sbtVersion),

    buildInfoOptions ++= { if (coverageEnabled.value) Seq() else Seq(BuildInfoOption.BuildTime) }, // <-- this line was changed
    buildInfoOptions += BuildInfoOption.ToJson,

    buildInfoPackage := "pme123.adapters.version"
  )

buildInfoOptions does not include BuildTime option when coverage is turned on.

It doesn't look elegeant, but it works. You can probably find better way.

Answer 2

instead of having different buildinfo objects depending on the phase, which could lead to compilation errors, you can use your own build time.

lazy val buildTime: SettingKey[String] = SettingKey[String]("buildTime", "time of build")

ThisBuild / buildTime := ZonedDateTime.now(ZoneOffset.UTC).toString

buildInfoKeys :=
  Seq[BuildInfoKey](
    name,
    version,
    scalaVersion,
    sbtVersion,
    buildTime
  )

This should resolve this issue. I have this configuration in a project of mine because I wanted a better control over the way the date is formatted, and I don't have the same issue