scalafor-comprehensionscalding

Why does a for comprehension expand to a `withFilter`


I'm working on a DSL for relational (SQL-like) operators. I have a Rep[Table] type with an .apply: ((Symbol, ...)) => Obj method that returns an object Obj which defines .flatMap: T1 => T2 and .map: T1 => T3 functions. As the type Rep[Table] does not know anything about the underlying table's schema, the apply method acts like a projection - projecting only the fields specified in the argument tuple (a lot like the untyped scalding api). Now type T1 is a "tuple-like", its length constrained to the projection tuple's length by some shapeless magic, but otherwise the types of the tuple elements are decided by the api user, so code like

val a = loadTable(...)
val res = a(('x, 'y)).map { (t: Row2[Int, Int]) =>
  (1, t(0))
}

or

val res = a(('x, 'y)).map { (t: Row2[String, String]) =>
  (1, t(0))
}

works fine. Note that the type of the argument for the map/flatMap function must be specified explicitly. However, when I try to use it with a for comprehension

val a = loadTable(...)
val b = loadTable(...)
val c = loadTable(...)

val res = for {
  as: Row2[Int, Int] <- a(('ax, 'ay))
  bs: Row2[Int, Int] <- b(('bx, 'by))
  cs: Row2[Int, Int] <- c(('cx, 'cy))
} yield (as(0), as(1), bs(0), bs(1), cs(0), cs(1))

it complains about the lack of a withFilter operator. Adding an .withFilter: T1 => Boolean does not cut it - it then complains about "missing parameter type for expanded function", as T1 is parameterised by some type. Only adding .withFilter: Row[Int, Int] => Boolean makes it work, but obviously is not what I want at all.

My questions are: why does the withFilter get called in the first place and how can I use it with my parameterised tuple-like type T1?


Edit In the end I went with a .withFilter: NothingLike => BoolLike which is a noop for simple checks like _.isInstanceOf[T1] and a more restricted .filter: T1 => BoolLike to be used in general case.


Solution

  • Unfortunately, you cannot use for-comprehensions when you expect type inference based on the argument type of your lambda.

    Actually, in your example, as: Row2[Int, Int] is interpreted as a pattern match:

    val res = for {
      as: Row2[Int, Int] <- a(('ax, 'ay))
    } yield (...)
    

    Translates to something like:

    a(('ax, 'ay)).withFilter(_.isInstanceOf[Row2[Int, Int]]).map(...)
    

    Pattern matching in for comprehensions can be very useful:

    val names = for {
      ("name", name) <- keyValPairs
    } yield name
    

    But the trade-off is, that you cannot explicitly specify the argument type of the lambda.