Logging vs. Debugging

Question

Background: I've inherited a web application that is intended to create on-the-fly connections between local and remote equipment. There are a tremendous number of moving parts recently: the app itself has changed significantly; the development toolchain was just updated; and both the local and remote equipment have been "modified" to support those changes.

The bright side is that it has a reasonable logging system that will write debug messages to a file, and it will log to both the file and a real-time user screen. I have an opportunity to re-work the entire log/debug mechanism.

Examples:

• All messages are time-stamped and prefixed with a severity level.
• Logs are for the customer. They record the system's responses to his/her requests.
• Any log that identifies a problem also suggests a solution.
• Debugs are for developers and Tech Support. They reveal the system internals.
• Debugs indicate the function and/or line that generated them.
• The customer can adjust the debug level on the fly to set the verbosity.

Question: What best practices have you used as a developer, or seen as a consumer, that generate useful logs and debugs?

---

Edit: Many helpful suggestions so far, thanks! To clarify: I'm more interested in what to log: content, format, etc.--and the reasons for doing so--than specific tools.

What was it about the best logs you've seen that made them most helpful?

Thanks for your help!

Answer 1

Don't confuse Logging, Tracing and Error Reporting, some people I know do and it creates one hell of a log file to grep through in order to get the information I want.

If I want to have everything churned out, I seperate into the following:

• Tracing -> Dumps every action and step, timestamped, with input and output data of that stage (ugliest and largest file)
• Logging -> Log the business process steps only, client does enquiry so log the enquiry criteria and output data nothing more.
• Error Reporting / Debugging -> Exceptions logged detailing where it occurred, timestamped, input/output data if possible, user information etc

That way if any errors occurred and the Error/Debug log doesn't contain enough information for my liking I can always do a grep -A 50 -B 50 'timestamp' tracing_file to get more detail.

EDIT: As has also been said, sticking to standard packages like the built in logging module for python as an example is always good. Rolling your own is not a great idea unless the language does not have one in it's standard library. I do like wrapping the logging in a small function generally taking the message and value for determining which logs it goes to, ie. 1 - tracing, 2 - logging, 4 - debugging so sending a value of 7 drops to all 3 etc.

Answer 2

The absolutley most valueable thing done with any logging framework is a "1-click" tool that gathers all logs and mail them to me even when the application is deployed on a machine belonging to a customer.

And make good choices at what to log so you can roughly follow the main paths in your application.

As frameworks I've used the standards (log4net, log4java, log4c++)

do NOT implement your own logging framework, when there already is a good one out-of-the-box. Most people who do just reinvent the wheel.