Erlang VM killed when creating millions of processes

Question

So after Joe Armstrongs' claims that erlang processes are cheap and vm can handle millions of them. I decided to test it on my machine:

process_galore(N)->
    io:format("process limit: ~p~n", [erlang:system_info(process_limit)]),
    statistics(runtime),
    statistics(wall_clock),
    L = for(0, N, fun()-> spawn(fun() -> wait() end) end),
    {_, Rt} = statistics(runtime),
    {_, Wt} = statistics(wall_clock),
    lists:foreach(fun(Pid)-> Pid ! die end, L),
    io:format("Processes created: ~p~n
          Run time ms: ~p~n
          Wall time ms: ~p~n
          Average run time: ~p microseconds!~n", [N, Rt, Wt, (Rt/N)*1000]).

wait()->
    receive die ->
         done
    end.

for(N, N, _)->
    [];
for(I, N, Fun) when I < N ->
    [Fun()|for(I+1, N, Fun)].

Results are impressive for million processes - I get aprox 6.6 micro! seconds average spawn time. But when starting 3m processes, OS shell prints "Killed" with erlang runtime gone. I run erl with +P 5000000 flag, system is: arch linux with quadcore i7 and 8GB ram.

Answer 1

Erlang processes are cheap, but they're not free. Erlang processes spawned by spawn use 338 words of memory, which is 2704 bytes on a 64 bit system. Spawning 3 million processes will use at least 8112 MB of RAM, not counting the overhead of creating the linked list of pids and the anonymous function created for each process (I'm not sure if they're shared if they're created like you're creating.) You'll probably need 10-12GB of free RAM to spawn and keep alive 3 million (almost) empty processes.

As I pointed out in the comments (and you later verified), the "Killed" message was printed by the Linux Kernel when it killed the Erlang VM, most likely for using up too much RAM. More information here.