Scala Future[Option[T]] Un Packing

Question

In the following snippet,

trait MyType1; trait MyType2
import scala.concurrent.Promise

val p1 = Promise[Option[MyType1]]()
val p2 = Promise[MyType2]()

I pass in p1 and p2 to another function where I complete the Promise using a successful Future. After the call to this function, I try to read the value in the Promise:

trait Test {
  // get the Future from the promise
  val f1 = p1.future
  val f2 = p2.future

  for {
    someF1Elem <- f1
    f1Elem     <- someF1Elem
    f2Elem     <- f1Elem
  } yield {
    // do something with f1Elem and f2Elem
    "..."
  }
}

When I try to compile this, I get some compiler issues.

Error:(52, 19) type mismatch;
 found   : Option[Nothing]
 required: scala.concurrent.Future[?]
      flElem     <- someF1Elem
                  ^

IntelliJ shows no errors or what-so-ever and the types look aligned. But I'm not sure why the compiler is unhappy! Any clues?

Answer 1

Your for comprehension types must be consistent, so you cannot freely mix Option and Future in the way you do.

In your example, f1 returns a Future[Option[MyType1]] while f2 returns a Future[MyType2]

Remember that a for comprehension desugars to a series of flatMap/map and potentially withFilter.

Also the (simplified) signatures of flatMap for Future[A] and Option[A] are

def flatMap[B](f: A => Future[B]): Future[B]
def flatMap[B](f: A => Option[B]): Option[B]

The first two steps of the for-comprehension desugar to

f1.flatMap { someF1Elem =>
  someF1Elem.flatMap { f1Elem => // this returns a `Option[MyType1]`
     ...
  }
}

see the error now?

---

Now, to fix this there's a few approaches you can follow. One very handy is to use Monad Transformers, which allow you to combine (for instance) Future and Option into a single monad type OptionT.

Specifically you can go back and forth from Future[Option[A]] to OptionT[Future, A]. The basic idea is that you can flatMap on the latter and extract a value of type A.

But before that, you need to make your types of the "right shape", so that both are a Future[Option[Something]]

Here's an example using scalaz

import scalaz._; import Scalaz._ ; import scalaz.OptionT._
import scala.concurrent.{ Promise, Future }
import scala.concurrent.ExecutionContext.Implicits.global

trait MyType1
trait MyType2

val p1 = Promise[Option[MyType1]]()
val p2 = Promise[MyType2]()

val f1 = p1.future
val f2 = p2.future 

val res = for {
  f1Elem <- optionT(f1)
  f2Elem <- f2.liftM[OptionT]
} yield {
  println(s"$f1Elem $f2Elem")
}

optionT builds an OptionT[Future, A] given a Future[Option[A]], while liftM[OptionT] achieves the same when you have a Future[A]

The type of res is OptionT[Future, Unit] in this case, and you can get a Future[Option[Unit]] by calling run on it.

Answer 2

Looking at "How does yield work", your for comprehension is equivalent to

f1.flatMap(someF1Elem: Option[MyType1] => 
  someF1Elem.flatMap(f1Elem => 
    f1Elem.map(f2Elem =>
      "..."
    )
  )
)

Basically what happens is this: flatMap on a Future is defined to take a function that returns another Future. This gives you a similar error:

<console>:64: error: value map is not a member of MyType1 f1Elem.map(f2Elem => ^ <console>:63: error: type mismatch; found : Option[Nothing] required: scala.concurrent.Future[?] someF1Elem.flatMap(f1Elem => ^

If you want the operation to perform asynchronously, i.e. really mapping the Future instances, you will have to get back to a Future[Option[X]]. As Kim suggested, you can nest the comprehension:

import scala.concurrent.{Future, Promise}

def test: Future[Option[String]] = {
  val p1 = Promise[Option[MyType1]]()
  val p2 = Promise[MyType2]()
  // ... run code that completes the promises

  val f1 = p1.future
  val f2 = p2.future

  for {
    someF1Elem: Option[MyType1] <- f1
    f2Elem     <- f2
  } yield {
    for (f1Elem <- someF1Elem) yield "something"
  }
}

You could also make the resulting Future fail when the option is not defined (NoSuchElementException)

def test: Future[String] = {
  val p1 = Promise[Option[MyType1]]()
  val p2 = Promise[MyType2]()
  // ... run code that completes the promises

  val f1 = p1.future
  val f2 = p2.future

  for {
    someF1Elem: Option[MyType1] <- f1
    f2Elem <- f2
  } yield {
    val f1Elem = someF1Elem.get  // may cause an exception and fail the future
    "something"
  }
}

Answer 3

In a for comprehension, the expressions on the right hand side of the <- have to have compatible types. That is because for comprehensions are basically syntactic sugar for nested flatMap calls. To solve this problem, you can either have a for comprehension nested within another for comprehension or you can use methods like get or map on the Options.

Another option is to use monad transformers, but that's beyond the scope of this answer.