Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why Seq.take in F# throw System.OutOfMemoryException

I have a code:

seq {for i in [1 .. 100000000] -> i} |> Seq.take 100000;;
Real: 00:00:00.000, CPU: 00:00:00.000, GC gen0: 0, gen1: 0, gen2: 0
val it : seq<int> =
  Error: Exception of type 'System.OutOfMemoryException' was thrown.

This code results in out of memory. Why? And why exception was thrown after time calculation (after operation was complete)? AFAIK, individual sequence elements are computed only as required?

like image 834
Sergey Avatar asked Aug 20 '13 12:08

Sergey


2 Answers

seq {for i in [1 .. 100000000] -> i} |> Seq.take 100000

Creates a sequence that will generate items as you ask for them, from the list [1 .. 100000000]. This requires the list to be in-memory. The list is too big for 32-bit memory, hence OutOfMemoryException. You should try

seq {for i in 1 .. 100000000 -> i} |> Seq.take 100000
like image 132
Ramon Snir Avatar answered Sep 21 '22 18:09

Ramon Snir


In addition to revealing the core problem reason it may be worth addressing the second part of the question as to why the exception is thrown after the operation is complete.

For getting to understand this it would help considering that

let mySeq = seq {for i in [1 .. 100000000] -> i} |> Seq.take 100000;;

will not be followed by any exceptions, while seemingly innocuous

seq {for i in [1 .. 100000000] -> i};;

will be followed by the same exception as in original, although we seemingly do not try materializing the sequence anyhow.

That's right, we do not, but FSI does, trying to print out values of few first sequence members, like for a smaller list below:

 seq {for i in [1 .. 100] -> i};;
 val it : seq<int> = seq [1; 2; 3; 4; ...]

which initiates the in-memory instantiation of the original bulky list .

like image 31
Gene Belitski Avatar answered Sep 23 '22 18:09

Gene Belitski