After a spawned Elixir process crashes, no other processes are spawned

Question

In my program, after each line from a CSV file is read, a new process is spawned to download that image and save it to the filesystem:

defmodule Downloader.CLI do
  alias Downloader.Parser
  alias Downloader.Getter

  def main(_args \\ []) do
    Enum.map(Parser.run, fn(line) ->
       line -> handle_download(line)
    end)
  end

  defp handle_download({ :ok, %{ "image_id" => image_id } }) do
    pid = spawn(Getter, :run, [])
    send(pid, {self(), image_id})

    receive do
      :ok -> nil
      err -> IO.inspect(err)
    end
  end
end

If this CSV file has 1000 images, then 1000 different elixir processes are being created in the VM. If only one of these processes throws an exception though, no other process continues. That is, the executable doesn't freeze, but no other images are downloaded.

Why does this happen? Why can't the other processes continue to execute, if they are independent from each other? I feel like missing something simple, but I just couldn't find it anywhere else.

Answer 1

Issue: The receive do waits for a message that never arrives.

This is what happens step by step:

1. spawn creates the process pid
2. process pid crashes, before sending back a message to the initial process
3. Initial process is stuck in receive function: it waits for a message. and as long as no message are sent, everything is on hold. Enum.map will not launch following download.

To illustrate this issue, run the following script in an iex:

Process.send_after(self(), "Hi Luis", 30000)
receive do
    mess -> IO.inspect mess
end

You will receive "Hi Luis" only after 30 seconds, and during those 30 seconds, your iex will be waiting for a message, because it is synchronous.

Solution: Make sure a message is always sent back, for example using Process.monitor/1 or a Supervisor.

defmodule Downloader.CLI do
  alias Downloader.Parser
  alias Downloader.Getter

  def main(_args \\ []) do
    Enum.map(Parser.run, fn(line) ->
       line -> handle_download(line)
    end)
  end

  defp handle_download({ :ok, %{ "image_id" => image_id } }) do
    pid = spawn(Getter, :run, [])
    Process.monitor(pid)
    send(pid, {self(), image_id})

    receive do
      :ok -> nil
      err -> IO.inspect(err)
    end
    flush   # To avoid having a Down message ending prematurely next download 
  end
end