Taming the Scala type system

Question

I don't seem to understand the Scala type system. I'm trying to implement two base traits and a trait for a family of algorithms to work with them. What am I doing wrong in the below?

The base traits for moves & states; these are simplified to just include methods that expose the problem.

trait Move
trait State[M <: Move] {
    def moves: List[M]
    def successor(m: M): State[M]
}

Here's the trait for the family of algorithms that makes use of above. I'm Not sure this is right! There might be some +M / -S stuff involved...

trait Algorithm {
    def bestMove[M <: Move, S <: State[M]](s: S): M
}

Concrete move and state:

case class MyMove(x: Int) extends Move
class MyState(val s: Map[MyMove,Int]) extends State[MyMove] {
    def moves = MyMove(1) :: MyMove(2) :: Nil
    def successor(p: MyMove) = new MyState(s.updated(p, 1))
}

I'm on very shaky ground regarding the below, but the compiler seems to accept it... Attempting to make a concrete implementation of the Algorithm trait.

object MyAlgorithm extends Algorithm {
    def bestMove(s: State[Move]) = s.moves.head
}

So far there are no compile errors; they show up when I try to put all the parts together, however:

object Main extends App {
    val s = new MyState(Map())
    val m = MyAlgorithm.bestMove(s)
    println(m)
}

The above throws this error:

error: overloaded method value bestMove with alternatives:
  (s: State[Move])Move <and>
  [M <: Move, S <: State[M]](s: S)M
 cannot be applied to (MyState)
    val m = MyAlgorithm.bestMove(s)
                        ^

Update: I changed the Algorithm trait to use abstract type members, as suggested. This solved the question as I had phrased it but I had simplified it a bit too much. The MyAlgorithm.bestMove() method must be allowed to call itself with the output from s.successor(m), like this:

trait Algorithm {
    type M <: Move
    type S <: State[M]
    def bestMove(s: S): M
}

trait MyAlgorithm extends Algorithm {
    def score(s: S): Int = s.moves.size
    def bestMove(s: S): M = {
        val groups = s.moves.groupBy(m => score(s.successor(m)))
        val max = groups.keys.max
        groups(max).head
    }
}

The above gives now 2 errors:

Foo.scala:38: error: type mismatch;
 found   : State[MyAlgorithm.this.M]
 required: MyAlgorithm.this.S
            val groups = s.moves.groupBy(m => score(s.successor(m)))
                                                               ^
Foo.scala:39: error: diverging implicit expansion for type Ordering[B]
starting with method Tuple9 in object Ordering
            val max = groups.keys.max
                                  ^

Do I have to move to an approach using traits of traits, aka the Cake pattern, to make this work? (I'm just guessing here; I'm thoroughly confused still.)

Answer 1

You declare MyAlgorithm#bestMove explicitly as taking a State[Move] parameter, but inside Main you are trying to pass it a MyState, which is a State[MyMove] not a State[Move].

You have a couple of options to resolve this. One would be to not constrain the types in MyAlgorithm:

object MyAlgorithm extends Algorithm {
    def bestMove[M <: Move, S <: State[M]](s: S) : M = s.moves.head
}

Unfortunately, the scala type inference isn't smart enough to figure these types out for you, so at the call site, you have to declare them, making the call to MyAlgorithm#bestMove look like this:

val m = MyAlgorithm.bestMove[MyMove, MyState](s)

Another option use abstract type members of the Algorithm trait:

trait Algorithm {
  type M <: Move
  type S <: State[M]
    def bestMove(s: S): M
}

And resolve the abstract types in the concrete implementation:

object MyAlgorithm extends Algorithm {
  type M = MyMove
  type S = MyState
  def bestMove(s: S) : M = s.moves.head
}

Then the call site gets to return to your original version, without mentioning the types:

val m = MyAlgorithm.bestMove(s)

You may want to keep MyAlgorithm unaware of the actual types, and leave the determining of those types to the 'clients' of that object, in which case, change the object to a trait:

trait MyAlgorithm extends Algorithm {
  def bestMove(s: S) : M = s.moves.head
}

Then in your Main class, you instantiate a MyAlgorithm with the concrete types:

val a = new MyAlgorithm {
  type M = MyMove
  type S = MyState
}
val m = a.bestMove(s)

Your comment "There might be some +M / -S stuff involved" was a good guess, but It won't work for you here. You might hope that the covariant type modifier "+" might help here. If you had declared the type parameter on State as

State[+M]

This would indicate that State[M] <:< State[N] if M <:< N. (read <:< as "is a subtype of"). Then you would have no problem passing in a State[MyMove] where a State[Move] was expected. However, you cannot use the covariant modifier on M here because it appears in the contravariant position as an argument to the successor function.

Why is this a problem? Your declaration of successor says that it will take an M and return a State. The covariant annotation says that a State[M] is also a State[Any]. So we should allow this assignment:

val x : State[Any] = y : State[MyMove]

Now if we have a State[Any], then x.successor is what type? Any => MyMove. Which cannot be correct, since your implementation is expecting a MyMove, not an Any

Answer 2

For updated code.

The compiler is very fair with complaints. Algorithm use one subclass of State as denoted and state successor may return any other subclass of State[M]

You may declare IntegerOne class

trait Abstract[T]
class IntegerOne extends Abstract[Int]

but the compiler have no clue that all instances of AbstractOne[Int] would be IntegerOne. It assume that there may be another class that also implements Abstract[Int]

class IntegerTwo extends Abstract[Int]

You may try to use implicit conversion to cast from Abstract[Int] to IntegerOne, but traits have no implicit view bounds as they have no value parameters at all.

Solution 0

So you may rewrite your Algorithm trait as an abstract class and use implicit conversion:

abstract class MyAlgorithm[MT <: Move, ST <: State[MT]] (implicit val toSM : State[MT] => ST) extends Algorithm {
  override type M = MT // finalize types, no further subtyping allowed
  override type S = ST // finalize types, no further subtyping allowed
  def score(s : S) : Int = s.moves.size
  override def bestMove(s : S) : M = {
    val groups = s.moves.groupBy( m => score(toSM ( s.successor(m)) ) )
    val max = groups.keys.max
    groups(max).head
  }
}

implicit def toMyState(state : State[MyMove]) : MyState = state.asInstanceOf[MyState]

object ConcreteAlgorithm extends MyAlgorithm[MyMove,MyState]

object Main extends App {
  val s = new MyState(Map())
  val m = ConcreteAlgorithm.bestMove(s)
  println(m)
}

There are two drawbacks in this solution

• using implicit conversion with asInstanceOf
• tying types

You may extinguish first as the cost of further type tying.

Solution 1

Let use Algorithm as sole type parameterization source and rewrite type structure accordingly

trait State[A <: Algorithm] { _:A#S =>
  def moves : List[A#M]
  def successor(m : A#M): A#S
}

trait Algorithm{
  type M <: Move
  type S <: State[this.type]
  def bestMove(s : S) : M
}

In that case your MyAlgorithm may be used without rewriting

trait MyAlgorithm extends Algorithm {
  def score(s : S) : Int = s.moves.size
  override def bestMove(s : S) : M = {
    val groups = s.moves.groupBy(m => score(s.successor(m)))
    val max = groups.keys.max
    groups(max).head
  }
}

Using it:

class MyState(val s : Map[MyMove,Int]) extends State[ConcreteAlgorithm.type] {
  def moves = MyMove(1) :: MyMove(2) :: Nil
  def successor(p : MyMove) = new MyState(s.updated(p,1))
}

object ConcreteAlgorithm extends MyAlgorithm {
  override type M = MyMove
  override type S = MyState
}

object Main extends App {
  val s = new MyState(Map())
  val m = ConcreteAlgorithm.bestMove(s)
  println(m)
}

See more abstract and complicated usage example for this tecnique: Scala: Abstract types vs generics

Solution 2

There is also a simple solution to your question but I doubt it can solve your problem. You will eventually stuck upon type inconsistency once again in more complex use cases.

Just make MyState.successor return this.type instead of State[M]

trait State[M <: Move] {
  def moves : List[M]
  def successor(m : M): this.type
}

final class MyState(val s : Map[MyMove,Int]) extends State[MyMove] {
  def moves = MyMove(1) :: MyMove(2) :: Nil
  def successor(p : MyMove) = (new MyState(s.updated(p,1))).asInstanceOf[this.type]
}

other things are unchanged

trait Algorithm{
  type M <: Move
  type S <: State[M]
  def bestMove(s : S) : M
}

trait MyAlgorithm extends Algorithm {
  def score(s : S) : Int = s.moves.size
  override def bestMove(s : S) : M = {
    val groups = s.moves.groupBy(m => score(s.successor(m)))
    val max = groups.keys.max
    groups(max).head
  }
}

object ConcreteAlgorithm extends MyAlgorithm {
  override type M = MyMove
  override type S = MyState
}

object Main extends App {
  val s = new MyState(Map())
  val m = ConcreteAlgorithm.bestMove(s)
  println(m)
}

Pay attention to final modifier to MyState class. It ensures that conversion asInstanceOf[this.type] is correct one. Scala compiler may compute itself that final class keeps always this.type but it still have some flaws.

Solution 3

There is no need to tie Algorithm with custom State. As long as Algorithm does not use specific State function it may be written simpler without type bounding exercises.

trait Algorithm{
  type M <: Move
  def bestMove(s : State[M]) : M
}
trait MyAlgorithm extends Algorithm {
  def score(s : State[M]) : Int = s.moves.size
  override def bestMove(s : State[M]) : M = {
    val groups = s.moves.groupBy(m => score(s.successor(m)))
    val max = groups.keys.max
    groups(max).head
  }
}

This simple example doesn't come to my mind quickly because I've assumed that binding to different states are obligatory. But sometimes only part of system really should be parameterized explicitly and your may avoid additional complexity with it

Conclusion

Problem discussed reflects bunch of problems that arises in my practice very often.

There are two competing purposes that should not exclude each other but do so in scala.

• extensibility
• generality

First means that you can build complex system, implement some basic realization and be able to replace its parts one by one to implement more complex realization.

Second allows your to define very abstract system, that may be used for different cases.

Scala developers had very challenging task for creating type system for a language that can be both functional and object oriented while being limited to jvm implementation core with huge defects like type erasure. Co/Contra-variance type annotation given to users are insufficient for expressing types relations in complex system

I have my hard times every time I encounter extensiblity-generality dilemma deciding which trade-off to accept.

I'd like not to use design pattern but to declare it in the target language. I hopes that scala will give me this ability someday.