How to Change log level for particular users/threads at runtime

Question

I'm using slf4j with either log4j 2.0 or logback as the implementation. For example, my servlet has a logger with level ERROR, and my server spawns 100 threads of the servlet. I will get a list of special users at runtime. When I detect some of the special users connected in. I want to change the log level for those special users/threads to DEBUG, and leave other threads' log level unaffected (still ERROR).

I know the TurboFilter in logback and DynamicThresholdFilter in log4j 2.0, but since I will only get the special users list at runtime, I cannot use them.

Here is my application:

package com.example.logging;

import java.util.HashMap;
import java.util.Map;

import javax.servlet.http.HttpServlet;

import org.slf4j.*;

public class App extends HttpServlet {

    private final Logger Logger = LoggerFactory.getLogger(App.class);
    Map<String, String> map = new HashMap<String, String>();

    public App() {
        map.put("user1", "DEBUG");
        map.put("user2", "DEBUG");
        map.put("user3", "ERROR");
    }

    public void writeToLogFile(String userName) {

        if (map.containsKey(userName)) {
            // do something so that I can change the logger to the corresponding log level
        }

        Logger.error(userName + " error message");

        // the logger is of level ERROR, so by default, this log event will not happen
        // but I want it to happen for special users
        if (Logger.isDebugEnabled()) {
            Logger.debug(userName + " debug message");
        }
    }
}

Here is my log configuration in log4j2.xml

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="ERROR">
<Appenders>
    <Console name="Console" target="SYSTEM_OUT">
        <PatternLayout pattern="%-5level %class{36} %M %msg%xEx%n" />
    </Console>
</Appenders>
<Loggers>
    <Logger name="com.example.logging.App" level="ERROR" additivity="false">
            <AppenderRef ref="Console" />
    </Logger>
    <Root level="DEBUG">
        <AppenderRef ref="Console" />
    </Root>
</Loggers> 
</Configuration>

If I call the methods below:

App myApp = new App();
// assume the below 4 methods are called concurrently
myApp.writeToLogFile("user1");
myApp.writeToLogFile("user2");
myApp.writeToLogFile("user3");
myApp.writeToLogFile("user4");

The expected output should be:

ERROR com.example.logging.App writeToLogFile - user1 error message
DEBUG com.example.logging.App writeToLogFile - user1 debug message
ERROR com.example.logging.App writeToLogFile - user2 error message
DEBUG com.example.logging.App writeToLogFile - user2 debug message
ERROR com.example.logging.App writeToLogFile - user3 error message
ERROR com.example.logging.App writeToLogFile - user4 error message

Answer 1

I've met the same problem, and I end up using my own filter by making changes to DynamicThresholdFilter

Changes to the application:

package com.example.logging;

import java.util.HashMap;
import java.util.Map;

import javax.servlet.http.HttpServlet;

import org.slf4j.*;

public class App extends HttpServlet {

    private final Logger Logger = LoggerFactory.getLogger(App.class);
    Map<String, String> map = new HashMap<String, String>();

    public App() {
        map.put("user1", "Debug");
        map.put("user2", "Debug");
        map.put("user3", "Error");
    }

    public void writeToLogFile(String userName) {
        // if the user is in the map, we put it into ThreadConext for filtering
        if (map.containsKey(userName)) {
            MDC.put("level", map.get(userName));
        }

        Logger.error(userName + " error message");

        if (Logger.isDebugEnabled()) {
            Logger.debug(userName + " debug message");
        }

            // remember to remove it
        MDC.remove("level");
    }

}

Here is the newly defined filter based on DynamicThresholdFilter, let's call it DynamicThresholdUserFilter, you can compare it to the source code of DynamicThresholdFilter

package com.example.logging.log4j2.plugin;

import org.apache.logging.log4j.Level;
import org.apache.logging.log4j.Marker;
import org.apache.logging.log4j.ThreadContext;
import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.core.Logger;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.config.plugins.PluginAttribute;
import org.apache.logging.log4j.core.config.plugins.PluginFactory;
import org.apache.logging.log4j.core.filter.AbstractFilter;
import org.apache.logging.log4j.message.Message;

/**
 * Compare against a log level that is associated with an MDC value.
 */
@Plugin(name = "DynamicThresholdUserFilter", category = "Core", elementType = "filter", printObject = true)
public final class DynamicThresholdUserFilter extends AbstractFilter {
    private Level defaultThreshold = Level.ERROR;
    private final String key;

    private DynamicThresholdUserFilter(final String key, final Level defaultLevel,
                                   final Result onMatch, final Result onMismatch) {
        super(onMatch, onMismatch);
        if (key == null) {
            throw new NullPointerException("key cannot be null");
        }
        this.key = key;
        this.defaultThreshold = defaultLevel;
    }

    public String getKey() {
        return this.key;
    }

    @Override
    public Result filter(final Logger logger, final Level level, final Marker marker, final String msg,
                         final Object... params) {
        return filter(level);
    }

    @Override
    public Result filter(final Logger logger, final Level level, final Marker marker, final Object msg,
                         final Throwable t) {
        return filter(level);
    }

    @Override
    public Result filter(final Logger logger, final Level level, final Marker marker, final Message msg,
                         final Throwable t) {
        return filter(level);
    }

    @Override
    public Result filter(final LogEvent event) {
        return filter(event.getLevel());
    }

    /* biggest change here */
    private Result filter(final Level level) {
        final String value = ThreadContext.get(key);
        if (value != null) {
            Level ctxLevel = Level.toLevel(value);
            if (ctxLevel == null) {
                // in case the level is invalid
                ctxLevel = defaultThreshold;
            }
            return level.isAtLeastAsSpecificAs(ctxLevel) ? onMatch : onMismatch;
        }
        return Result.NEUTRAL;

    }

    @Override
    public String toString() {
        final StringBuilder sb = new StringBuilder();
        sb.append("key=").append(key);
        sb.append(", default=").append(defaultThreshold);
        return sb.toString();
    }

    /**
     * Create the DynamicThresholdFilter.
     * @param key The name of the key to compare.
     * @param pairs An array of value and Level pairs.
     * @param levelName The default Level.
     * @param match The action to perform if a match occurs.
     * @param mismatch The action to perform if no match occurs.
     * @return The DynamicThresholdFilter.
     */
    @PluginFactory
    public static DynamicThresholdUserFilter createFilter(
            @PluginAttribute("key") final String key,
            @PluginAttribute("defaultThreshold") final String levelName,
            @PluginAttribute("onMatch") final String match,
            @PluginAttribute("onMismatch") final String mismatch) {
        final Result onMatch = Result.toResult(match);
        final Result onMismatch = Result.toResult(mismatch);
        final Level level = Level.toLevel(levelName, Level.ERROR);
        return new DynamicThresholdUserFilter(key, level, onMatch, onMismatch);
    }
}

Add the DynamicThresholdUserFilter and package name to your configuration file

<?xml version="1.0" encoding="UTF-8"?>
<!-- add the package name of the filter-->
<Configuration status="ERROR" packages="com.example.logging.plugin">
    <!-- configuration of the new defined filter -->
    <DynamicThresholdUserFilter key="level" defaultThreshold="ERROR" onMatch="ACCEPT" onMismatch="NEUTRAL" />
    <Appenders>
        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout pattern="%-5level %class{36} %M %msg%xEx%n"/>
        </Console>
    </Appenders>
    <Loggers>
        <Logger name="com.example.logging.App" level="ERROR" additivity="false">
            <AppenderRef ref="Console" />
        </Logger>
        <Root level="debug">
            <AppenderRef ref="Console" />
        </Root>
    </Loggers>
</Configuration>

The newly defined filter is pretty similar to DynamicThresholdFilter. The difference is DynamicThresholdFilter uses the predefined level in configuration file as the dynamic threshold, while this filter uses the level programmatically defined in the map.

Answer 2

While the already existing answer might work (haven't tried it personally), after intensive searching, I found a very easy and neat trick to do what you are requesting.

The DynamicThresholdFilter can be used with conditions to switch the log level at run time. This, combined with log4j2's ThreadContext, you can do quite nifty things.

You would have to populate a particular key in the ThreadContext at the beginning of a server call processing (somewhere in doFilter method of your HttpServlet class) based on your custom logic of user names. This would look something like:

ThreadContext.put("customLogLevel", "debug");

Then in your log4j2.xml file, you put this as a global filter, right below the root Configuration tag:

<DynamicThresholdFilter key="customLogLevel" onMatch="ACCEPT" onMismatch="NEUTRAL">
    <KeyValuePair key="debug" value="DEBUG"/>
    <KeyValuePair key="error" value="ERROR"/>
    <KeyValuePair key="info" value="INFO"/>
</DynamicThresholdFilter>

Now based on the value of the key customLogLevel in the ThreadContext that you set at the beginning of a call, all the log calls in that thread will have log level corresponding to the matching KeyValuePair line. So in the example above, all log calls in the thread would have level as DEBUG.