Shapeless not finding implicits in test, but can in REPL

Question

I have a case class that looks like this:

case class Color(name: String, red: Int, green: Int, blue: Int)

I'm using Shapeless 2.3.1 with Scala 2.11.8. I'm seeing different behavior from my test and the REPL in terms of finding the implicit value for LabelledGeneric[Color]. (I'm actually trying to auto-derive some other typeclass, but I'm getting null for that too)

Inside test

package foo

import shapeless._
import org.specs2.mutable._

case class Color(name: String, red: Int, green: Int, blue: Int)

object CustomProtocol {
  implicit val colorLabel: LabelledGeneric[Color] = LabelledGeneric[Color]
}

class GenericFormatsSpec extends Specification {
  val color = Color("CadetBlue", 95, 158, 160)

  "The case class example" should {
    "behave as expected" in {
      import CustomProtocol._
      assert(colorLabel != null, "colorLabel is null")
      1 mustEqual 1
    }
  }
}

This test fails because colorLabel is null. Why?

REPL

From the REPL, I can find LabelledGeneric[Color]:

scala> case class Color(name: String, red: Int, green: Int, blue: Int)
defined class Color

scala> import shapeless._
import shapeless._

scala> LabelledGeneric[Color]
res0: shapeless.LabelledGeneric[Color]{type Repr = shapeless.::[String with shapeless.labelled.KeyTag[Symbol with shapeless.tag.Tagged[String("name")],String],shapeless.::[Int with shapeless.labelled.KeyTag[Symbol with shapeless.tag.Tagged[String("red")],Int],shapeless.::[Int with shapeless.labelled.KeyTag[Symbol with shapeless.tag.Tagged[String("green")],Int],shapeless.::[Int with shapeless.labelled.KeyTag[Symbol with shapeless.tag.Tagged[String("blue")],Int],shapeless.HNil]]]]} = shapeless.LabelledGeneric$$anon$1@755f11d9

Answer 1

The null you're seeing is, indeed, a surprising consequence of the semantics of implicit definitions with and without explicitly annotated types. The expression on the right hand side of the definition, LabelledGeneric[Color], is a call of the apply method on object LabelledGeneric with type argument Color which itself requires an implicit argument of type LabelledGeneric[Color]. The implicit lookup rules imply that the corresponding in-scope implicit definition with the highest priority is the implicit val colorLabel which is currently under definition, ie. we have a cycle which ends up with the value getting the default null initializer. If, OTOH, the type annotation is left off, colorLabel isn't in scope, and you'll get the result that you expect. This is unfortunate because, as you rightly observe, we should explicitly annotate implicit definitions wherever possible.

shapeless's cachedImplicit provides a mechanism for solving this problem, but before describing it I need to point out one additional complication. The type LabelledGeneric[Color] isn't the right type for colorLabel. LabelledGeneric has a type member Repr which is the representation type of the type you're instantiating the LabelledGeneric for, and by annotating the definition as you have you are explicitly discarding the refinement of LabelledGeneric[Color] which includes that. The resulting value would be useless because its type isn't sufficiently precise. Annotating the implicit definition with the correct type, either with an explicit refinement or using the equivalent Aux is difficult because the representation type is complex to write out explicitly,

object CustomProtocol {
  implicit val colorLabel: LabelledGeneric.Aux[Color, ???] = ...
}

Solving both of these problems simultaneously is a two step process,

• obtain the LabelledGeneric instance with the fully refined type.
• define the cached implicit value with an explict annotation but without generating an init cycle resulting in a null.

That ends up looking like this,

object CustomProtocol {
  val gen0 = cachedImplicit[LabelledGeneric[Color]]
  implicit val colorLabel: LabelledGeneric.Aux[Color, gen0.Repr] = gen0
}

Answer 2

I just realized that I was putting return type for the implicit:

object CustomProtocol {
  implicit val colorLabel: LabelledGeneric[Color] = LabelledGeneric[Color]
}

but the actual return type in the REPL is something like

shapeless.LabelledGeneric[Color]{type Repr = shapeless.::[String with shapeless.labelled.KeyTag[Symbol with shapeless.tag.Tagged[String("name")],String],shapeless.::[Int with shapeless.labelled.KeyTag[Symbol with shapeless.tag.Tagged[String("red")],Int],shapeless.::[Int with shapeless.labelled.KeyTag[Symbol with shapeless.tag.Tagged[String("green")],Int],shapeless.::[Int with shapeless.labelled.KeyTag[Symbol with shapeless.tag.Tagged[String("blue")],Int],shapeless.HNil]]]]}

The test passes when I remove the type annotation:

object CustomProtocol {
  implicit val colorLabel = LabelledGeneric[Color]
}

This is surprising, since normally we are encouraged to put type annotation for the implicits.