Elixir/ExUnit: passing context from testcase to teardown/cleanup method (on_exit) possible?

Question

Problem

I want to test an Elixir module that interacts with the host system and has methods that have side effects. For this question and to keep it brief, assume it is the creation of several directories. Those directories should of course be deleted after the tests are run, and also if the tests (which are pretty long) fail for any reasons (bad module code, bad test code, etc.).

I would like to know how to best/most elegantly solve this cleanup step. I have looked at the documentation of ExUnit.Callbacks.on_exit/2, but its examples are only for setup and simple teardown (no passed state involved). I have also searched online, but found nothing useful, so it could be that my idea itself is not good - I am also open to suggestions to reframe the problem.

Example

defmodule SimpleTest do
  use ExUnit.Case

  setup_all do
    ts = Time.utc_now |> Time.to_string
    {:ok, [timestamp: ts]}
    # paths to be cleaned are not yet known here
  end

  test "first test", context do
    path = "/tmp/dir" <> context[:timestamp]
    assert :ok == SimpleModule.mkdir(path)
    assert :eexist == SimpleModule.mkdir(path)
    # [path] should be passed on to cleanup
  end

  test "second test", context do
    path = "/tmp/dir" <> context[:timestamp]
    path2 = "/tmp/dir2" <> context[:timestamp]
    SimpleModule.mkdir(path)
    SimpleModule.mkdir(path2)
    assert File.exists?(path)
    assert File.exists?(path2)
    # [path, path2] should be passed on to cleanup
  end

  defp cleanup(context) do
    Enum.each(context[:dirs], fn(x) -> File.rmdir(x) end)
  end
end

defmodule SimpleModule do
  def mkdir(path) do
    File.mkdir(path)
  end
end

Possible solutions?

I now want to add a call to cleanup/1 with a list of directories to delete after each tests. The following ideas are things that I have tried:

• Calling the function directly at the end of each test: works for simple cases, but if the test case loops endlessly, it is killed and the cleanup does not happen anymore.
• Calling on_exit(fn -> cleanup(context) end) with updated context inside each test: this seems to work, but I could not find out if it is recommended and if it makes a difference where to put the call inside the test (beginning/end).
• Calling on_exit(fn -> cleanup(context) end) in the setup context function: The documentation does this, but I don't know how to pass any new state/context to it. It seems to only be useful if all context is already completely defined in the setup functions.

Maybe I am also overthinking this problem... I just had some bad debugging experiences with incomplete cleanup and resulting endless recursion (which should have been caught by my code, but was not yet), so I just want to make sure I do the right thing and learn it the correct way. Aside from those tests, Elixir is a very pleasant and flawless experience so far!

Answer 1

In this particular case I would just register the on_exit callback in you setup function (your third solution).

Instead of deleting the paths individually, delete the parent directory:

@test_dir "/tmp/base_test"

setup do
  File.mkdir(@test_dir)

  on_exit fn ->
    File.rm_rf @test_dir
  end
end

And then use @test_dir as your base directory in your tests

Answer 2

You can also register a callback to be executed on test exit inside your test case and pass it the specific path.

test "first test", context do
    path = "/tmp/dir" <> context[:timestamp]

    on_exit(fn -> cleanup(path) end)

    assert :ok == SimpleModule.mkdir(path)
    assert :eexist == SimpleModule.mkdir(path)
end

test "second test", context do
    path = "/tmp/dir" <> context[:timestamp]
    path2 = "/tmp/dir2" <> context[:timestamp]
    SimpleModule.mkdir(path)
    SimpleModule.mkdir(path2)
    assert File.exists?(path)
    assert File.exists?(path2)
    on_exit(fn -> cleanup(path) end)
    on_exit(fn -> cleanup(path) end)
end

You can register it at any point of your test case, it will be executed after the test has ended. You can also register them with a reference term.

As explained in ExUnit docs:

on_exit/2 callbacks are registered on demand, usually to undo an action performed by a setup callback. on_exit/2 may also take a reference, allowing callback to be overridden in the future. A registered on_exit/2 callback always runs, while failures in setup and setup_all will stop all remaining setup callbacks from executing.