Scala slow builds: development approaches to avoid

Question

First of all, incremental builds via SBT are pretty awesome, generally in the < 1sec range. However, sometimes you have to do a full clean/compile, or, in the case of incremental builds, you make a change to one file which then triggers the compilation of dozens of other files.

This is when Scala development becomes less...fun, as the resulting slowdown in work flow can encourage context switching (check email, latest Stackoverflow threads, etc.), which subtly makes one less productive

So, what are the development approaches to avoid in order to improve full clean/compile builds, and (ideally), change-one-file-without-recompiling-half-the-application incremental builds?

Examples I can think of:
1) is it better to have a thousand+ line do-it-all scala file, or several files split up?
2) can I have my cake (pattern) or will that inflate build times?
3) can I have pimp'd x,y,z library pattern, or better to find another way?
4) are package objects (with implicits) a build time killer?
5) nested objects and traits?
6) implicit methods/parameters or stop being clever and be explicit?

Concretely, I'm thinking of ditching a cake pattern DAO I came up with and consolidating into ScalaQuery case class + companion object + minimal database provider trait. That alone will shed 20 scala files.

The application is small enough (120 scala + 10 java files) to refactor now without too much hassle. Obviously as a scala application grows, so too will the build times, just based on LOCs alone. I'm just trying to see where to trim the fat and where not to bother (i.e. keep things as they are) so current and future applications benefit from the expressiveness that scala affords without needlessly inflating build times.

Thanks for some examples of your experience of the good, the bad, and the ugly of scala development vis a vis build times.

Answer 1

I've noticed that type members can force rebuilds in places you would not expect. For example:

foo.scala:

object foo {
    class A {
        type F = Float
    }
    def z: Int = 8
}

bar.scala:

object bar {
    def run { println(foo.z) }
}

Changing the value of z does not force bar to be recompiled. Changing the type of F does, even though bar never refers to F or even to A. Why, I have no idea (Scala 2.9.1).

Answer 2

Have a look at how incremental recompilation works in SBT.

It's roughly this:

1. Find all classes whose publicly visible API have changed
2. Invalidate all of its dependents, its dependents' dependents, and so on.

For the purposes of SBT, a "dependent" is both a user of the class and a class defined in the same file.

Owen's example for foo.scala could even be this, and you'd see the issue:

object foo {
  def z: Int = 8
}

object foo2 {
  class A { ... }
}

Good practices:

• Separate files for separate classes
• Fine-grained interfaces
• Use the same level of abstraction in companion objects as in their companion classes; if the companion object reaches up through layers of abstraction, pull it into a separate class and file.