Scaladoc generation fails when referencing methods generated by annotation macros

Question

I have two classes, call them Foo and Fizz. Foo uses an annotation macro called expand to make aliases of some of its methods (the actual implementation does a little more than creating aliases, but the simple version still exhibits the problem that follows). For simplicity, let's say the expand macro simply takes all of the methods within the annotated class, and makes a copy of them, appending "Copy" to the end of the method name, then forwarding the call to the original method.

My problem is that if I use the expand macro on Foo, which creates a copy of a method Foo#bar called barCopy, when barCopy is called within another class, Fizz, everything compiles but scaladoc generation fails, as such:

[error] ../src/main/scala/Foo.scala:11: value barCopy is not a member of Foo
[error]     def str = foo.barCopy("hey")
[error]                   ^
[info] No documentation generated with unsuccessful compiler run

If I remove the scaladoc that tags the method being copied (Foo#bar), the sbt doc command works again. It's as if the scaladoc generator invokes an early phase of the compiler without using the macro paradise plugin that's enabled, yet somehow it works if the docs are removed from the offending method.

This is the expand macro:

import scala.annotation.{ StaticAnnotation, compileTimeOnly }
import scala.language.experimental.macros
import scala.reflect.macros.whitebox.Context

@compileTimeOnly("You must enable the macro paradise plugin.")
class expand extends StaticAnnotation {
    def macroTransform(annottees: Any*): Any = macro Impl.impl
}

object Impl {

  def impl(c: Context)(annottees: c.Expr[Any]*): c.Expr[Any] = {
    import c.universe._

    val result = annottees map (_.tree) match {
      case (classDef @
        q"""
          $mods class $tpname[..$tparams] $ctorMods(...$paramss) extends { ..$earlydefns } with ..$parents {
            $self => ..$stats
          }
        """) :: _ =>

        val copies = for {
            q"def $tname[..$tparams](...$paramss): $tpt = $expr" <- stats
            ident = TermName(tname.toString + "Copy")
        } yield {
            val paramSymbols = paramss.map(_.map(_.name))
            q"def $ident[..$tparams](...$paramss): $tpt = $tname(...$paramSymbols)"
        }
        q"""
            $mods class $tpname[..$tparams] $ctorMods(...$paramss) extends { ..$earlydefns } with ..$parents { $self =>
                ..$stats
                ..$copies
            }
        """
        case _ => c.abort(c.enclosingPosition, "Invalid annotation target: not a class")
    }

    c.Expr[Any](result)
  }

}

And the classes, which exist in a separate project:

/** This is a class that will have some methods copied. */
@expand class Foo {
    /** Remove this scaladoc comment, and `sbt doc` will run just fine! */
    def bar(value: String) = value
}

/** Another class. */
class Fizz(foo: Foo) {
    /** More scaladoc, nothing wrong here. */
    def str = foo.barCopy("hey")
}

This seems like a bug, or perhaps a missing feature, but is there a way to generate scaladoc for the above classes without removing docs from copied methods? I've tried this with both Scala 2.11.8 and 2.12.1. This is a simple sbt project that demonstrates the issue I'm having.

Answer 1

This is a bug in Scala, still present in 2.13. This gist of the issue is that when compiling for Scaladoc (as with sbt doc), the compiler introduces extra DocDef AST nodes for holding comments. Those are not matched by the quasiquote pattern. Even worse, they aren't even visible from the scala-reflect API.

Here is an excerpt from a comment by @driuzz explaining the situation on a similar issue in simulacrum:

[...] During normal compilation methods are available as DefDef type even when they have scaladoc comment which is just ignored. But during sbt doc compiler generates a little different AST. Every method that has scaladoc comment is described as DocDef(comment, DefDef(...)) which causes that macro isn't recognizing them at all [...]

The fix that @driuzz implemented is here. The idea is to try casting the scala-reflect trees into their Scala compiler representations. For the code from the question, this means defining some unwrapDocDef to help remove the docstrings from the methods.

    val result = annottees map (_.tree) match {
      case (classDef @
        q"""
          $mods class $tpname[..$tparams] $ctorMods(...$paramss) extends { ..$earlydefns } with ..$parents {
            $self => ..$stats
          }
        """) :: _ =>

        // If the outer layer of the Tree is a `DocDef`, peel it back
        val unwrapDocDef = (t: Tree) => {
          import scala.tools.nsc.ast.Trees

          if (t.isInstanceOf[Trees#DocDef]) {
            t.asInstanceOf[Trees#DocDef].definition.asInstanceOf[Tree]
          } else {
            t
          }
        }

        val copies = for {
            q"def $tname[..$tparams](...$paramss): $tpt = $expr" <- stats.map(unwrapDocDef)
            ident = TermName(tname.toString + "Copy")
        } yield {
            val paramSymbols = paramss.map(_.map(_.name))
            q"def $ident[..$tparams](...$paramss): $tpt = $tname(...$paramSymbols)"
        }
        q"""
            $mods class $tpname[..$tparams] $ctorMods(...$paramss) extends { ..$earlydefns } with ..$parents { $self =>
                ..$stats
                ..$copies
            }
        """
        case _ => c.abort(c.enclosingPosition, "Invalid annotation target: not a class")
    }

Of course, since this imports something from the Scala compiler, the SBT definition of the macro project has to change:

lazy val macros = (project in file("macros")).settings(
    name := "macros",
    libraryDependencies ++= Seq(
        "org.scala-lang" % "scala-reflect" % scalaV,
        "org.scala-lang" % "scala-compiler" % scalaV  // new
    )
).settings(commonSettings: _*)