What is a first class module in Scala?

Question

What does it mean when it is said that scala provides for first class module support through the object syntax? Nothing in the glossary even mentions the phrase but I've run into it twice now and haven't been able to decipher it. Is was said in this blog post regarding adapters.

Answer 1

A "module" is a pluggable piece of software, also called a "package" sometimes. It provides a functionality through a well-defined set of interfaces, which declare what it provides and what it requires. Finally, it is interchangeable.

There are few languages with direct support for modules, mostly because while support for declaring APIs is common, support for declaring dependencies or requirements is not. Libraries in popular languages will usually interface by relying on types provided by the "standard" library, or requiring initialization with objects implementing APIs they provide.

So, if I want to make a benchmark module, I'll usually resort to clock facilities provided by the standard libraries or, at worse, I'll declare a clock type and request to be initialized with a class implementing it before the module's functionality is ready to be used.

When module support is available, however, I'll not only declare the benchmark interfaces I provide, but I'll also declare that I need a "clock module" -- a module exporting certain interfaces that I need.

A client of my module would not be required to do anything to use my interfaces -- it could just go ahead and use it. Or it could not even declare that my benchmark module would be used and, instead, declare that it has a requirement for a benchmark module.

What will satisfy the requirements is only decided at the "top" level, at the application level (modules are components of applications). At that point it would declare that it would use that client, my benchmark, and a module implementing my clock requirements.

If you know Guice, this might seem familiar. The problems that Guice addresses are in large part caused by the lack of module support in the Java programming language.

So, back to Scala. How does module support work? Well, the interface of my module might look like this:

trait Benchmark extends Clock // What I need {
  // What I provide
  type ABench <: Bench
  trait Bench {
    def measure(task: => Unit): Long
  }
  def aBench: ABench
}

and Clock would be a module definition, such as this:

trait Clock {
  // What it provides
  type AClock <: Clock
  trait Clock {
    def now(): Long
  }
  def aClock: AClock
}

My module itself might look like this:

trait MyModule extends Benchmark {
  class ABench extends Bench {
    def measure(task: => Unit): Long = {
      val measurements = for(_ <- 1 to 10) yield {
        val start = aClock.now()
        task
        val end = aClock.now()
        end - start
      }
      measurements / 10
    }
  }
  object aBench extends ABench
}

The Clock module would be similarly defined. An application could be declared as being a composition of modules:

trait application extends Clock with Benchmark with ...

though, of course, dependencies need not be declared, as they are already provided for. You can then combine modules that provide the requirements to build the application:

object Application extends MyModule with JavaClock with ...

And that would link the requirements of MyModule with the implementation provided by JavaClock. The stuff above still require some shared knowledge, because "Clock" is likely, an API that I provide. One can write a proxy, of course, but it's not plug and play. Scala can go a bit further, if I declared my Benchmark module like this:

trait Benchmark {
  type Clock = {
    def now(): Long
  }
  def aClock: Clock

  // What I provide
  type ABench <: Bench
  trait Bench {
    def measure(task: => Unit): Long
  }
  def aBench: ABench
}

Now any class that offers a now(): Long method can be used to satisfy the requirement, without any bridge. Of course, if the name of the methods is "millis(): Long" instead of "now(): Long", I'm still screwed, and that kind of "binding" is something that languages providing module support might address, though not Scala. Also, due to how JVM works, there's a performance penalty there as well.

So, that's the module, and the module support. Finally first class module. First class support for X means X can be manipulated as values. For example, Scala has first class support for functions, which means I can pass a function to a method, store it in a variable, in a map, etc.

The first class support for modules, is basically instantiation, though one can use an "object" to create a singleton of that module, and then pass that around (the advantages of which I discuss further below). So, I can do this:

object myBenchmark extends MyModule with JVMClock

and pass myBenchmark as a parameter to methods that need such a module.

There are two elements in Scala that make all of that work: abstract types and path dependent types.

Abstract types, the "type" declarations, make it possible for a piece of code to declare it will be using a type X, which is not going to be defined by who calls or instantiates it, but, rather, at the moment modules get composed.

Path dependent types make it possible to work with modules without being completely unsafe, but without being so restrictive as to not allow anything. Let's say I do this:

val b: MyModule.Clock = MyModule.aClock

And let's say I have a method on Benchmark that took a Clock as a parameter. I can call that method on MyModule passing b as a parameter, because Scala knows that the Clock of b is the clock bound to MyModule. And if I try to pass b to another module implementing Benchmark, Scala would not let me do that. That is, I can get values out of Benchmark that are specific to abstract types of Benchmark -- unknown to all but the module implementations -- and feed it back to that Benchmark but not other Benchmark implementations.