Nested implicit macros seems not working for type parameters?

Question

I'm writing a Scala implicit macro that automatically generates a type class for case classes (using quasiquote, both Scala 2.10.3 with the macro paradise compiler plugin and Scala 2.11.0-M7).

The implicit macro recursively looks up type classes for the parameter.

As long as the case class does not take type parameters or the type parameters are not used in the code generated, it works fine.

But once implicit value of <TypeClass>[<TypeParameter of case class>] is required, compilation of the call site fails with "could not find implicit value for parameter e".

Here is the code that reproduces the issue:

trait TestTypeClass[A] {
  def str(x: A): String
}
object Test {
  implicit def BooleanTest = new TestTypeClass[Boolean] {
    def str(x: Boolean) = x.toString
  }
  def CaseClassTestImpl[A: c.WeakTypeTag](c: Context): c.Expr[TestTypeClass[A]] = {
    import c.universe._
    val aType = weakTypeOf[A]
    val TestTypeClassType = weakTypeOf[TestTypeClass[_]]
    val typeName = aType.typeSymbol.name.decoded
    val params = aType.declarations.collectFirst { case m: MethodSymbol if m.isPrimaryConstructor => m }.get.paramss.head
    val paramTypes = aType.declarations.collectFirst { case m: MethodSymbol if m.isPrimaryConstructor => m }.get.paramss.head.map(_.typeSignature)
    val paramList = for (i <- 0 until params.size) yield {
      val param = params(i)
      val paramType = paramTypes(i)
      val paramName = param.name.decoded
      q"str($param)"
    }
    println(paramList)
    val src =
      q"""
    new TestTypeClass[$aType] {
      def str(x: $aType) = Seq(..$paramList).mkString(",")
    }
    """
    c.Expr[TestTypeClass[A]](src)
  }
  implicit def CaseClassTest[A]: TestTypeClass[A] = macro CaseClassTestImpl[A]
  def str[A: TestTypeClass](x: A) = implicitly[TestTypeClass[A]].str(x)
}

// somewhere in other module
implicitly[TestTypeClass[TestClass]] // OK.
implicitly[TestTypeClass[TestClass2[Boolean]]] // Error
// could not find implicit value for parameter e: TestTypeClass[TestClass2[Boolean]]
implicitly[TestTypeClass[TestClass2[TestClass]]] // Error
// could not find implicit value for parameter e: TestTypeClass[TestClass2[TestClass]]

Is it so by design, am I doing something wrong, or is it a compiler bug?

Answer 1

There are some surface-level problems that should prevent your version from working at all, but once they're addressed you should be able to do exactly what you want (whether it's a good idea or not is another question—one that I'll try to address at the end of this answer).

The three biggest problems are in this line:

      q"str($param)"

First of all, in the context of the generated code, str is going to refer to the method on the anonymous class you're defining and instantiating, not to the str method on Test. Next, this will generate code that looks like str(member), but member won't mean anything in the context of the generated code—you want something like str(x.member). Lastly (and relatedly), each param is going to be a constructor parameter, not an accessor.

The following is a complete working example (tested on 2.10.3):

import scala.language.experimental.macros
import scala.reflect.macros.Context

trait TestTypeClass[A] { def str(x: A): String }

object Test {
  implicit def BooleanTest = new TestTypeClass[Boolean] {
    def str(x: Boolean) = x.toString
  }

  def CaseClassTestImpl[A: c.WeakTypeTag](
    c: Context
  ): c.Expr[TestTypeClass[A]] = {
    import c.universe._
    val aType = weakTypeOf[A]

    val params = aType.declarations.collect {
      case m: MethodSymbol if m.isCaseAccessor => m
    }.toList

    val paramList = params.map(param => q"Test.str(x.$param)")

    val src = q"""
      new TestTypeClass[$aType] {
        def str(x: $aType) = Seq(..$paramList).mkString(",")
      }
    """

    c.Expr[TestTypeClass[A]](src)
  }
  implicit def CaseClassTest[A]: TestTypeClass[A] = macro CaseClassTestImpl[A]
  def str[A: TestTypeClass](x: A) = implicitly[TestTypeClass[A]].str(x)
}

And then some demonstration setup:

import Test._

case class Foo(x: Boolean, y: Boolean)
case class Bar[A](a: A)

And finally:

scala> str(Bar(Foo(true, false)))
res0: String = true,false

Which shows us that the compiler has successfully found the instance for Bar[Foo] by applying the macro recursively.

So this approach works, but it also undermines some of the big advantages that type classes provide over e.g. runtime reflection-based solutions to this kind of problem. It becomes much less easy to reason about what instances are available when we've got some macro that's just pulling them out of the air. The logic that determines what it's able to find is buried in the macro implementation code—which will be run at compile-time, so it's still type-safe in a sense, but it's less transparent.

This implementation also wildly over-generates instances (try str(1)), which could pretty easily be corrected, but it's a good illustration of how dangerous this kind of stuff can be.

For what it's worth, the following is an alternative solution using Shapeless 2.0's TypeClass type class, mentioned by Miles above (you can also see my blog post here for a similar comparison).

implicit def BooleanTest = new TestTypeClass[Boolean] {
  def str(x: Boolean) = x.toString
}

def str[A: TestTypeClass](x: A) = implicitly[TestTypeClass[A]].str(x)

import shapeless._

implicit object `TTC is a type class` extends ProductTypeClass[TestTypeClass] {
  def product[H, T <: HList](htc: TestTypeClass[H], ttc: TestTypeClass[T]) =
    new TestTypeClass[H :: T] {
      def str(x: H :: T) = {
        val hs = htc.str(x.head)
        val ts = ttc.str(x.tail)
        if (ts.isEmpty) hs else hs + "," + ts
      }
    }
  def emptyProduct = new TestTypeClass[HNil] { def str(x: HNil) = "" }
  def project[F, G](inst: => TestTypeClass[G], to: F => G, from: G => F) =
    new TestTypeClass[F] { def str(x: F) = inst.str(to(x)) }
}

object TestTypeClassHelper extends TypeClassCompanion[TestTypeClass]
import TestTypeClassHelper.auto._

It's not really any more concise, but it's more generic and less likely to do something you don't expect. There's still magic happening, but it's easier to control and reason about.