How to use trace and dbg in Erlang to debug and trace my program?

Question

I am trying to start using erlang:trace/3 and the dbg module to trace the behaviour of a live production system without taking the server down.

The documentation is opaque (to put it mildly) and there don't appear to be any useful tutorials online.

What I spent all day trying to do was capture what was happening in a particular function by trying to apply a trace to Module:Function using dbg:c and dbg:p but with no success at all.

Does anyone have a succinct explanation of how to use trace in a live Erlang system?

Answer 1

The basic steps of tracing for function calls are on a non-live node:

> dbg:start().   % start dbg > dbg:tracer().  % start a simple tracer process > dbg:tp(Module, Function, Arity, []).   % specify MFA you are interested in > dbg:p(all, c).   % trace calls (c) of that MFA for all processes.  ... trace here  > dbg:stop_clear().   % stop tracer and clear effect of tp and p calls. 

You can trace for multiple functions at the same time. Add functions by calling tp for each function. If you want to trace for non-exported functions, you need to call tpl. To remove functions, call ctp or ctpl in a similar manner. Some general tp calls are:

> dbg:tpl(Module, '_', []).  % all calls in Module > dbg:tpl(Module, Function, '_', []).   % all calls to Module:Function with any arity. > dbg:tpl(Module, Function, Arity, []). % all calls to Module:Function/Arity. > dbg:tpl(M, F, A, [{'_', [], [{return_trace}]}]).   % same as before, but also show return value. 

The last argument is a match specification. You can play around with that by using dbg:fun2ms.

You can select the processes to trace on with the call to p(). The items are described under erlang:trace. Some calls are:

> dbg:p(all, c).   % trace calls to selected functions by all functions > dbg:p(new, c).   % trace calls by processes spawned from now on > dbg:p(Pid, c).   % trace calls by given process > dbg:p(Pid, [c, m]).  % trace calls and messages of a given process 

I guess you will never need to directly call erlang:trace, as dbg does pretty much everything for you.

A golden rule for a live node is to generate only an amount of trace output to the shell, which lets you to type in dbg:stop_clear().. :)

I often use a tracer that will auto-stop itself after a number of events. For example:

dbg:tracer(process, {fun (_,100) -> dbg:stop_clear();                         (Msg, N) -> io:format("~p~n", [Msg]), N+1 end, 0                     }). 

If you are looking for debugging on remote nodes (or multiple nodes), search for pan, eper, inviso or onviso.