Where can I learn about constructing AST's for Scala macros?

Question

Where I can learn how to construct the AST's that Scala's macros generate?

The Scaladoc isn't as helpful as I'd like. For example:

abstract def Apply(sym: Universe.Symbol, args: Universe.Tree*): Universe.Tree A factory method for Apply nodes. 

But how do I figure out what an Apply node is? Where can I find a list of the node types in AST's, and how they fit together?

Answer 1

There isn't a lot of documentation for the internals of the compiler available, but the things that are available should be enough to get started.

Mirko Stocker, has written his Master Thesis about Scala Refactoring. In Appendix D (p. 95) he describes the architecture of the AST. It includes also a graphical overview:

Another way to find information about the AST is to look directly into the sources of reflect.internal.Trees, which contains the AST.

If one needs to find out how a specific source code snippet is represented internally there is reify:

scala> import reflect.runtime.universe._ import reflect.runtime.universe._  scala> showRaw(reify{val i = 0}.tree) res8: String = Block(List(ValDef(Modifiers(), newTermName("i"), TypeTree(),   Literal(Constant(0)))), Literal(Constant(()))) 

Answer 2

You could take a look at the scaladoc (http://docs.scala-lang.org/overviews/reflection/symbols-trees-types.html#trees) or at the slides (http://scalamacros.org/talks/2012-04-28-MetaprogrammingInScala210.pdf, the "Learn to learn" part).

Here's what I usually do. I wrote a simple script called parse, which takes Scala code as an argument and then compiles it with -Xprint:parser -Ystop-after:parser -Yshow-trees-stringified -Yshow-trees-compact (parse uses another helper script: adhoc-scalac. click here to see its sources as well).

The advantage this approach has over showRaw is that it doesn't require the code to typecheck. You could write a small snippet of code, which refers to non-existent variables or classes, and it still will successfully run and show you the AST. Here's an example of output:

09:26 ~$ parse 'class C { def x = 2 }' [[syntax trees at end of parser]]// Scala source: tmp36sVGp package <empty> {   class C extends scala.AnyRef {     def <init>() = {       super.<init>();       ()     };     def x = 2   } } PackageDef(Ident(TermName("<empty>")), List(ClassDef(Modifiers(), TypeName("C"), List(), Template(List(Select(Ident(scala), TypeName("AnyRef"))), emptyValDef, List(DefDef(Modifiers(), nme.CONSTRUCTOR, List(), List(List()), TypeTree(), Block(List(pendingSuperCall), Literal(Constant(())))), DefDef(Modifiers(), TermName("x"), List(), List(), TypeTree(), Literal(Constant(2)))))))) 

There's also a script called typecheck, which does the same, but stops after typer. That's sometimes useful to understand how exactly the typechecker transforms the parser trees. However, both toolboxes and macros work with parser trees, so I use typecheck for tree construction purposes very rarely.