Java Logging: log everything >debug but only when an error occured

Question

During a request processing there are bunch of debug logs logged. At some point there might an error be logged if problem occurs. Idea: when no error hapens during whole request processing I would like to ignore all debugs, BUT if error happens, I would like to see in logs all preceeding and succeding debugs.

Is there any out-of-the-box solution for that (in log4j, logback or any other), or must I implement some kind of wrapper for my Logger, that will "cache" debugs per Thread until they are needed?

Answer 1

It is not a built in solution but here it mys draft implementation what could is quite minimal and easy to integrate. It isusing log4j2 and there is no need to change your client code if you are already using log4j2 or slf4j.

So the corresponding test case

  @Test
  public void no_error_so_no_log() throws Exception {
    LogEventCollector.clean();
    try {
      log.debug("hello world");
      log.debug("some debug again");
    } finally {
      LogEventCollector.clean();
    }
  }

  @Test
  public void error_so_log_from_the_beginning() throws Exception {
    LogEventCollector.clean();
    try {
      log.debug("hello world");
      log.error("some error", new RuntimeException("whatever"));
      log.debug("some debug again");
    } finally {
      LogEventCollector.clean();
    }
  }

LogEventCollector will take care of initialize and the cleanup on you web app request thread. It could be put into a servlet Filter.

I have created my custom Appender. The core of the appender is the following:

    @Override
    public void append(LogEvent event) {
      if (Level.ERROR.isLessSpecificThan(event.getLevel()))
        LogEventCollector.markError();
      if (!LogEventCollector.hadError()) {
        LogEventCollector.collect(event);
        return;
      }
      for (LogEvent collected : LogEventCollector.events())
        push_log_out(collected);
      //and the current one
      push_log_out(event);
    }

Method push_log_event is the real logging. It could your custom implementation or a delegate to an other one (like AsyncAppender).

the LogEventCollector itself:

  public class LogEventCollector {
    static ThreadLocal<Context> LOG_COLLECTOR = new ThreadLocal<Context>() {
      @Override
      protected Context initialValue() {
        return new Context();
      }
    };

    public static void clean() {
      LOG_COLLECTOR.get().clean();
    }

    static class Context {
      boolean had_error = false;
      List<LogEvent> events = new ArrayList<LogEvent>();

      public void clean() {
        had_error = false;
        events = new ArrayList<LogEvent>();
      }
    }

    public static void markError() {
      LOG_COLLECTOR.get().had_error = true;
    }

    public static boolean hadError() {
      return LOG_COLLECTOR.get().had_error;
    }

    public static void collect(LogEvent event) {
      LOG_COLLECTOR.get().events.add(event);
    }

    public static List<LogEvent> events() {
      List<LogEvent> ret = LOG_COLLECTOR.get().events;
      LOG_COLLECTOR.get().events = new ArrayList<LogEvent>();
      return ret;
    }

  }

I think it must be enough to customize for your own needs.

Answer 2

You are talking about ringbuffer logging, also known as backtrace logging.

You can implement this in Java using terse-logback's ringbuffer feature.

<ringBuffer name="JDBC_RINGBUFFER">
    <capacity>${jdbc.ringBuffer.capacity}</capacity>
</ringBuffer>

and then dumping the ring buffer happens by logging to a control appender:

<logger name="JDBC_RINGBUFFER_LOGGER" level="TRACE" additivity="false">
    <!-- This appender dumps contents of the ring buffer when an event is received. -->
    <appender class="com.tersesystems.logback.ringbuffer.DumpRingBufferAppender">
        <!-- Event source -->
        <ringBuffer-ref ref="JDBC_RINGBUFFER"/>

        <!-- Event source -->
        <appender-ref ref="ASYNC_JDBC"/>
    </appender>
</logger>

Ringbuffer logging can be very useful, but does have some drawbacks from only being in memory and only a list. There's another appender to SQLite, Blacklite, that lets you keep the SQLite instance only in memory, and that way you can more easily query it.

Answer 3

I believe what you are asking for is captured in https://issues.apache.org/jira/browse/LOG4J2-1137.