Can I "pimp my library" with an analogue of TraversableLike.map that has nicely variant types?

Question

Suppose I want to add functionality like map to a Scala List, something along the lines of list mapmap f, which applies the function f to each element of list twice. (A more serious example might be implementing a parallel or distributed map, but I don't want to get distracted by details in that direction.)

My first approach would be

object MapMap {
    implicit def createFancyList[A](list: List[A]) = new Object {
        def mapmap(f: A => A): List[A] = { list map { a: A => f(f(a)) } }
    }
}

this now works great

scala> import MapMap._
import MapMap._

scala> List(1,2,3) mapmap { _ + 1 }
res1: List[Int] = List(3, 4, 5)

except of course it's only for Lists, and there's no reason we shouldn't want this to work for anything Traverseable, with a map function, e.g. Sets or Streams. So the second attempt looks like

object MapMap2 {
    implicit def createFancyTraversable[A](t: Traversable[A]) = new Object {
        def mapmap(f: A => A): Traversable[A] = { t map { a: A => f(f(a)) } }
    }
}

But now, of course, the result can't be assigned to a List[A]:

scala> import MapMap2._
import MapMap2._

scala> val r: List[Int] = List(1,2,3) mapmap { _ + 1 }
<console>:9: error: type mismatch;
 found   : Traversable[Int]
 required: List[Int]

Is there some middle ground? Can I write an implicit conversion that adds a method to all subclasses of Traversable, and successfully returns objects with that type?

(I'm guess this involves understanding the dreaded CanBuildFrom trait, and maybe even breakout!)

Answer 1

You can't do this for all Traversables, as they don't guarantee that map returns anything more specific than Traversable. See Update 2 below.

import collection.generic.CanBuildFrom
import collection.TraversableLike

class TraversableW[CC[X] <: TraversableLike[X, CC[X]], A](value: CC[A]) {
  def mapmap(f: A => A)(implicit cbf: CanBuildFrom[CC[A], A, CC[A]]): CC[A] 
      = value.map(f andThen f)
  def mapToString(implicit cbf: CanBuildFrom[CC[A], String, CC[String]]): CC[String]
      = value.map(_.toString)
}

object TraversableW {
  implicit def TraversableWTo[CC[X] <: TraversableLike[X, CC[X]], A](t: CC[A]): TraversableW[CC, A] 
      = new TraversableW[CC, A](t)
}

locally {
  import TraversableW._

  List(1).mapmap(1+)
  List(1).mapToString
  // The static type of Seq is preserved, *and* the dynamic type of List is also
  // preserved.
  assert((List(1): Seq[Int]).mapmap(1+) == List(3))
}

UPDATE I've added another pimped method, mapToString, to demonstrate why TraversableW accepts two type parameters, rather than one parameter as in Alexey's solution. The parameter CC is a higher kinded type, it represents the container type of the original collection. The second parameter, A, represents the element type of the original collection. The method mapToString is thus able to return the original container type with a different element type: CC[String.

UPDATE 2 Thanks to @oxbow_lakes comment, I've rethought this. It is indeed possible to directly pimp CC[X] <: Traversable[X], TraversableLike is not strictly needed. Comments inline:

import collection.generic.CanBuildFrom
import collection.TraversableLike

class TraversableW[CC[X] <: Traversable[X], A](value: CC[A]) {
  /**
   * A CanBuildFromInstance based purely the target element type `Elem`
   * and the target container type `CC`. This can be converted to a
   * `CanBuildFrom[Source, Elem, CC[Elem]` for any type `Source` by
   * `collection.breakOut`.
   */
  type CanBuildTo[Elem, CC[X]] = CanBuildFrom[Nothing, Elem, CC[Elem]]

  /**
   * `value` is _only_ known to be a `Traversable[A]`. This in turn
   * turn extends `TraversableLike[A, Traversable[A]]`. The signature
   * of `TraversableLike#map` requires an implicit `CanBuildFrom[Traversable[A], B, That]`,
   * specifically in the call below `CanBuildFrom[Traversable[A], A CC[A]`.
   *
   * Essentially, the specific type of the source collection is not known in the signature
   * of `map`.
   *
   * This cannot be directly found instead we look up a `CanBuildTo[A, CC[A]]` and
   * convert it with `collection.breakOut`
   *
   * In the first example that referenced `TraversableLike[A, CC[A]]`, `map` required a
   * `CanBuildFrom[CC[A], A, CC[A]]` which could be found.
   */
  def mapmap(f: A => A)(implicit cbf: CanBuildTo[A, CC]): CC[A]
      = value.map[A, CC[A]](f andThen f)(collection.breakOut)
  def mapToString(implicit cbf: CanBuildTo[String, CC]): CC[String]
      = value.map[String, CC[String]](_.toString)(collection.breakOut)
}

object TraversableW {
  implicit def TraversableWTo[CC[X] <: Traversable[X], A](t: CC[A]): TraversableW[CC, A]
      = new TraversableW[CC, A](t)
}

locally {
  import TraversableW._

  assert((List(1)).mapmap(1+) == List(3))

  // The static type of `Seq` has been preserved, but the dynamic type of `List` was lost.
  // This is a penalty for using `collection.breakOut`. 
  assert((List(1): Seq[Int]).mapmap(1+) == Seq(3))   
}

What's the difference? We had to use collection.breakOut, because we can't recover the specific collection subtype from a mere Traversable[A].

def map[B, That](f: A => B)(implicit bf: CanBuildFrom[Repr, B, That]): That = {
  val b = bf(repr)
  b.sizeHint(this) 
  for (x <- this) b += f(x)
  b.result
}

The Builder b is initialized with the original collection, which is the mechanism to preserve the dynamic type through a map. However, our CanBuildFrom disavowed all knowledge of the From, by way of the type argument Nothing. All you can do with Nothing is ignore it, which is exactly what breakOut does:

def breakOut[From, T, To](implicit b : CanBuildFrom[Nothing, T, To]) =
  new CanBuildFrom[From, T, To] {
    def apply(from: From) = b.apply();
    def apply() = b.apply()
  }

We can't call b.apply(from), no more than you could call def foo(a: Nothing) = 0.

Answer 2

As a general rule, when you want to return objects with the same type, you need TraversableLike (IterableLike, SeqLike, etc.) instead of Traversable. Here is the most general version I could come up with (the separate FancyTraversable class is there to avoid inferring structural types and the reflection hit):

class FancyTraversable[A, S <: TraversableLike[A, S]](t: S) {
  def mapmap(f: A => A)(implicit bf: CanBuildFrom[S,A,S]): S = { t map { a: A => f(f(a)) } }
}

implicit def createFancyTraversable[A, S <: TraversableLike[A, S]](t: S): FancyTraversable[A, S] = new FancyTraversable(t)