Scala streams and their memory usage

Question

As I understand, Stream retains the recently evaluated elements. I guess it does not retain all evaluated elements (it is not feasible), so it probably uses some internal "cache".

Is it correct? Can I control the size and policies of this cache?

Answer 1

Streams are like lists that generate their members as they are required. Once an element has been generated, it is retained in the stream and reused.

For example:

lazy val naturals: Stream[Int] = Stream.cons(0, naturals.map{_ + 1})

will give you a stream of the natural numbers. If I call

naturals(5)

it will generate elements 0-5 and return 5, if I then call

naturals(8)

It will reuse the first 6 elements and generate 3 more.

If you are concerned about memory usage, you can use Stream.drop(num) to produce a new stream with num elements removed from the beginning, allowing the truncated elements to be garbage collected with the old stream. For example:

naturals(5) //returns 5
val truncated = naturals.drop(4)
truncated(5) //returns 9

Answer 2

The Stream-object retains all references that have been evaluated/accessed so far. Stream works like a List. Every element that can be reached from a held reference, and which has already been accessed at least once, won't be garbage collected.

So basically your pointers into the stream and what you have evaluated so far define what will get cached.