withFilter instead of filter

Question

Is it always more performant to use withFilter instead of filter, when afterwards applying functions like map, flatmap etc.?

Why are only map, flatmap and foreach supported? (Expected functions like forall/exists as well)

Answer 1

From the Scala docs:

Note: the difference between c filter p and c withFilter p is that the former creates a new collection, whereas the latter only restricts the domain of subsequent map, flatMap, foreach, and withFilter operations.

So filter will take the original collection and produce a new collection, but withFilter will non-strictly (i.e. lazily) pass unfiltered values through to later map/flatMap/withFilter calls, saving a second pass through the (filtered) collection. Hence it will be more efficient when passing through to these subsequent method calls.

In fact, withFilter is specifically designed for working with chains of these methods, which is what a for comprehension is de-sugared into. No other methods (such as forall/exists) are required for this, so they have not been added to the FilterMonadic return type of withFilter.

Answer 2

In addition of the excellent answer of Shadowlands, I would like to bring an intuitive example of the difference between filter and withFilter.

Let's consider the following code

val list = List(1, 2, 3) var go = true val result = for(i <- list; if(go)) yield {    go = false    i } 

Most people expect result to be equal to List(1). This is the case since Scala 2.8, because the for-comprehension is translated into

val result = list withFilter {   case i => go } map {   case i => {     go = false     i   } } 

As you can see the translation converts the condition into a call to withFilter. Prior Scala 2.8, for-comprehension were translated into something like the following:

val r2 = list filter {   case i => go } map {   case i => {     go = false     i   } } 

Using filter, the value of result would be fairly different: List(1, 2, 3). The fact that we're making the go flag false has no effect on the filter, because the filter is already done. Again, in Scala 2.8, this issue is solved using withFilter. When withFilter is used, the condition is evaluated every time an element is accessed inside a map method.

Reference: - p.120 ,Scala in action (covers Scala 2.10), Manning Publications, Milanjan Raychaudhuri - Odersky's thoughts about for-comprehension translation