What's the recommended way to pass the results of macro computations to run-time?

Question

I'm trying to build some SQL-like abstraction and I have hit a problem.

This is a simplified “database table”:

trait Coffee {
  def id: Long
  def name: String
  def brand: String
}

This is my query abstraction:

import language.experimental.macros

object Query {  
  def from[T] = 
    macro QueryMacros.fromMacro[T]
}

class From[T] {  
  def select[S](s: T => S): Select[T] =
    macro QueryMacros.selectMacro[T, S]
}

class Select[T] {
  def where(pred: T => Boolean): Where =
    macro QueryMacros.whereMacro[T]
}

class Where(val result: String)

This is my macro implementation:

import scala.reflect.macros.Context

object QueryMacros {
  val result = new StringBuilder

  def fromMacro[T : c.WeakTypeTag](c: Context): c.Expr[From[T]] = { 
    result ++= ("FROM " + c.weakTypeOf[T])
    c.universe.reify(new From[T])
  }

  def selectMacro[T : c.WeakTypeTag, S : c.WeakTypeTag](c: Context)(s: c.Expr[T => S]): c.Expr[Select[T]] = {
    result ++= ("SELECT " + s.tree)
    c.universe.reify(new Select[T])
  }

  def whereMacro[S](c: Context)(pred: c.Expr[S]): c.Expr[Where] = {
    result ++= ("WHERE " + pred.tree)
    c.universe.reify(new Where(result.toString))
  }
}

And this is my example code:

object Main extends App {
  println("Query start")
  val query = 
    Query.from[Coffee]
         .select(_.id)
         .where(_.brand == "FairTrade")

  println(query.result)
  println("Query end")
}

It compiles and runs fine, but the output is:

Query start

Query end

Basically, result seems to be empty. I expected that it would hold the accumulated strings of the trees.

How can I pass my data from the macro compile stage to the next stage, so it shows up at runtime? I could of course pass the current string to the next method explicitly, but I would like to avoid that.

Answer 1

Basically you need to have a Queryable abstraction that: 1) provides the collection API (from, select, etc), 2) remembers the methods that were called on it by reifying the calls and accumulating them inside.

This concept is somewhat explained in our ScalaDays slides [1] and is implemented in Slick (which is open source) [2]. By the way in LINQ they do roughly the same with methods on Queryable reifying the calls and feeding them to your object that implements IQueryable, e.g. as described in [3].

Links:

1. http://scalamacros.org/talks/2012-04-18-ScalaDays2012.pdf
2. https://github.com/slick/slick/tree/master/src/main/scala/scala/slick/queryable
3. http://community.bartdesmet.net/blogs/bart/archive/2007/04/06/the-iqueryable-tales-linq-to-ldap-part-1-key-concepts.aspx

Answer 2

The problem is not passing the information from one macro call to the next one. All of those happen at compile time, so that should work. The problem is with the macro that is called last. Since it returns c.universe.reify(new Where(result.toString)), new Where(result.toString) is called at runtime. And then result will be empty. What you can do is return c.Expr(tree), where tree applies Where's constructor to a String literal that contains result.toString.

Also, you should note that your code depends on the order in which the macro calls are compiled. If you have several calls to these macros in several code files, result might contains information from previous calls. It would probably be best to rethink your whole approach.