Handling the cleanup of the gen_server state

Question

I have a gen_server running which it must clean up its state whenever it is stopped normally or it crash unexpectedly. The cleanup basically consists in deleting a few files.

At this moment, when the gen_server crash or it is stopped normally, the cleanup is done in terminate/2.

Is there any reason why terminate/2 would not be called if the gen_server crash?

Should be any other process monitoring the gen_server waiting to do the cleanup if the gen_server dies unexpectedly?

So, the code is like this:

terminate(normal, State) ->
    % Invoked when the process stops
    % Clean up the mess
terminate(Error, State) ->
    % Invoked when the process crashes
    % Clean up the mess

---

EDIT: I found this email in the official mailing list which is talking about the same thing:

http://groups.google.com/group/erlang-programming/browse_thread/thread/9a1ba2d974775ce8

As Adam says below, if we want to avoid to trap the exists in the gen_server, we could use different approaches.

But if we trap the exists, terminate/2 seems to be a safe place to do the cleanup as it always will be called. Furthermore we must handle correctly when 'EXIT' is sent to terminate/2 and to handle_call/3 trying to propagate the errors correctly between workers and supervisors.

Answer 1

terminate/2 is called when a crash occur inside the gen_server even if it doesn't trap exits, it will not be called if it receives an 'EXIT' from some other process linked to it, in case you need to clean up then it should trap exits(using process_flag(trap_exit, true)).

This behavior is a bit unfortunate because it makes it difficult to write a reliable shutdown procedure for a gen_server process. Also, it is not a good habit to trap exits just for the sake of being able to run terminate/2, since you might catch a lot of other errors which makes it harder to debug the system.

I would consider three options:

1. Handle the left over files when the next instance of the process starts (for example, in init/1)
2. Trap exits, clean up the files, and then crash again with the same reason
3. Have a 3rd process which monitors the gen_server whose only purpose is to clean up the files

Option 1 is probably the best option, since at least the code doesn't trap exits and you get persistent state for free. Option 2 is not so nice for the reasons described above, that it can hide and obscure other errors. 3 is messy because the cleanup process might not be done before the gen_server is started again.

Think carefully about why you want to clean up, and if it really has to be done when the process crashes (it is a bug, after all). Be careful that you don't end up doing too much defensive programming.