Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

When are scala's for-comprehensions lazy?

In Python, I can do something like this:

lazy = ((i,j) for i in range(0,10000) for j in range(0,10000))
sum((1 for i in lazy))

It will take a while, but the memory use is constant.

The same construct in scala:

(for(i<-0 to 10000; j<-i+1 to 10000) yield (i,j)).count((a:(Int,Int)) => true)

After a while, I get a java.lang.OutOfMemoryError, even though it should be evaluated lazily.

like image 548
tstenner Avatar asked Aug 09 '11 12:08

tstenner


1 Answers

Nothing's inherently lazy about Scala's for-comprehension; it's syntactic sugar* which won't change the fact that the combination of your two ranges will be eager.

If you work with lazy views of your ranges, the result of the comprehension will be lazy too:

scala> for(i<-(0 to 10000).view; j<-(i+1 to 10000).view) yield (i,j)
res0: scala.collection.SeqView[(Int, Int),Seq[_]] = SeqViewN(...)

scala> res0.count((a: (Int, Int)) => true)
res1: Int = 50005000

The laziness here is nothing to do with the for-comprehension, but because when flatMap or map (see below) are called on some type of container, you get back a result in the same type of container. So, the for-comprehension will just preserve the laziness (or lack of) of whatever you put in.


*for something like:

(0 to 10000).flatMap(i => (i+1 to 10000).map(j => (i, j)))
like image 88
Ben James Avatar answered Sep 28 '22 05:09

Ben James