Why does a for comprehension expand to a `withFilter`

Question

I'm working on a DSL for relational (SQL-like) operators. I have a Rep[Table] type with an .apply: ((Symbol, ...)) => Obj method that returns an object Obj which defines .flatMap: T1 => T2 and .map: T1 => T3 functions. As the type Rep[Table] does not know anything about the underlying table's schema, the apply method acts like a projection - projecting only the fields specified in the argument tuple (a lot like the untyped scalding api). Now type T1 is a "tuple-like", its length constrained to the projection tuple's length by some shapeless magic, but otherwise the types of the tuple elements are decided by the api user, so code like

val a = loadTable(...)
val res = a(('x, 'y)).map { (t: Row2[Int, Int]) =>
  (1, t(0))
}

or

val res = a(('x, 'y)).map { (t: Row2[String, String]) =>
  (1, t(0))
}

works fine. Note that the type of the argument for the map/flatMap function must be specified explicitly. However, when I try to use it with a for comprehension

val a = loadTable(...)
val b = loadTable(...)
val c = loadTable(...)

val res = for {
  as: Row2[Int, Int] <- a(('ax, 'ay))
  bs: Row2[Int, Int] <- b(('bx, 'by))
  cs: Row2[Int, Int] <- c(('cx, 'cy))
} yield (as(0), as(1), bs(0), bs(1), cs(0), cs(1))

it complains about the lack of a withFilter operator. Adding an .withFilter: T1 => Boolean does not cut it - it then complains about "missing parameter type for expanded function", as T1 is parameterised by some type. Only adding .withFilter: Row[Int, Int] => Boolean makes it work, but obviously is not what I want at all.

My questions are: why does the withFilter get called in the first place and how can I use it with my parameterised tuple-like type T1?

---

Edit In the end I went with a .withFilter: NothingLike => BoolLike which is a noop for simple checks like _.isInstanceOf[T1] and a more restricted .filter: T1 => BoolLike to be used in general case.

Answer 1

Unfortunately, you cannot use for-comprehensions when you expect type inference based on the argument type of your lambda.

Actually, in your example, as: Row2[Int, Int] is interpreted as a pattern match:

val res = for {
  as: Row2[Int, Int] <- a(('ax, 'ay))
} yield (...)

Translates to something like:

a(('ax, 'ay)).withFilter(_.isInstanceOf[Row2[Int, Int]]).map(...)

Pattern matching in for comprehensions can be very useful:

val names = for {
  ("name", name) <- keyValPairs
} yield name

But the trade-off is, that you cannot explicitly specify the argument type of the lambda.

Answer 2

I bumped into this issue also. Thanks to gzm0 explaining the Scala compilers behaviour I came up with this workaround:

import cats._
import cats.data._
import cats.implicits._

object CatsNEL extends App {
  val nss: NonEmptyList[(Int, String)] = NonEmptyList.of((1,"a"), (2, "b"), (3, "c"))
  val ss: NonEmptyList[String] = for {
    tuple <- nss
    (n, s) = tuple
  } yield s
}