Unable to use for comprehension to map over List within Future

Question

I have this issue that I have to work around every time. I can't map over something that is contained within a Future using a for comprehension.

Example:

import scala.concurrent.ExecutionContext.Implicits.global import scala.concurrent.Future  val f = Future( List("A", "B", "C") ) for {   list <- f   e <- list } yield (e -> 1) 

This gives me the error:

 error: type mismatch;  found   : List[(String, Int)]  required: scala.concurrent.Future[?]               e <- list                 ^ 

But if I do this it works fine:

f.map( _.map( (_ -> 1) ) ) 

Should i not be able to do this by using a for comprehension, is the reason it works in my other example that I do not flatmap? I'm using Scala 2.10.0.

Answer 1

Well, when you have multiple generators in a single for comprehension, you are flattening the resulting type. That is, instead of getting a List[List[T]], you get a List[T]:

scala> val list = List(1, 2, 3) list: List[Int] = List(1, 2, 3)  scala> for (a <- list) yield for (b <- list) yield (a, b) res0: List[List[(Int, Int)]] = List(List((1,1), (1,2), (1,3)), List((2,1 ), (2,2), (2,3)), List((3,1), (3,2), (3,3)))  scala> for (a <- list; b <- list) yield (a, b) res1: List[(Int, Int)] = List((1,1), (1,2), (1,3), (2,1), (2,2), (2,3), (3,1), (3,2), (3,3)) 

Now, how would you flatten a Future[List[T]]? It can't be a Future[T], because you'll be getting multiple T, and a Future (as opposed to a List) can only store one of them. The same problem happens with Option, by the way:

scala> for (a <- Some(3); b <- list) yield (a, b) <console>:9: error: type mismatch;  found   : List[(Int, Int)]  required: Option[?]               for (a <- Some(3); b <- list) yield (a, b)                                    ^ 

The easiest way around it is to simply nest multiple for comprehensions:

scala> for {      |   list <- f      | } yield for {      |   e <- list      | } yield (e -> 1) res3: scala.concurrent.Future[List[(String, Int)]] = scala.concurrent.im pl.Promise$DefaultPromise@4f498585 

In retrospect, this limitation should have been pretty obvious. The problem is that pretty much all examples use collections, and all collections are just GenTraversableOnce, so they can be mixed freely. Add to that, the CanBuildFrom mechanism for which Scala has been much criticized makes it possible to mix in arbitrary collections and get specific types back, instead of GenTraversableOnce.

And, to make things even more blurry, Option can be converted into an Iterable, which makes it possible to combine options with collections as long as the option doesn't come first.

But the main source of confusion, in my opinion, is that no one ever mentions this limitation when teaching for comprehensions.

Answer 2

Hmm, I think I got it. I need to wrap within a future as the for comprehension adds a flatmap.

This works:

for {   list <- f   e <- Future( list ) } yield (e -> 1) 

---

When I added above I did not see any answers yet. However to expand on this it is possible to do work within one for-comprehension. Not sure if it is worth the Future overhead though (edit: by using successful there should be no overhead).

for {   list1 <- f   list2 <- Future.successful( list1.map( _ -> 1) )   list3 <- Future.successful( list2.filter( _._2 == 1 ) ) } yield list3 

---

Addendum, half a year later.

Another way to solve this is to simply use assignment = instead of <- when you have another type than the initial map return type.

When using assignment that line does not get flat-mapped. You are now free to do an explicit map (or other transformation) that returns a different type.

This is useful if you have several transformation where one step that does not have the same return type as the other steps, but you still want to use the for-comprehension syntax because it makes you code more readable.

import scala.concurrent.ExecutionContext.Implicits.global import scala.concurrent.Future  val f = Future( List("A", "B", "C") ) def longRunning( l:List[(String, Int)] ) = Future.successful( l.map(_._2) )  for {   list <- f   e = list.map( _ -> 1 )   s <- longRunning( e ) } yield s