Logback and JsonLayout: can't pass custom fields

Question

I have this configuration in my logback.xml into a Spring Web Application (NO Spring Boot).

<?xml version="1.0" encoding="UTF-8"?>
<configuration>

     <appender name="Console" class="ch.qos.logback.core.ConsoleAppender">

        <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">             
            <layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
                <timestampFormat>yyyy-MM-dd'T'HH:mm:ss.SSSX</timestampFormat>
                <timestampFormatTimezoneId>Etc/UTC</timestampFormatTimezoneId>              
                <jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
                    <prettyPrint>true</prettyPrint>
                </jsonFormatter>
            </layout>            
            <customFields>{"appname":"foobar"}</customFields>            
        </encoder>                    
    </appender>

    <!-- LOG everything at INFO level -->
    <root level="INFO">
        <appender-ref ref="Console" />
    </root>

</configuration>

The JSON layout works fine but custom fields as "appname": "foobar" are not printed:

{
  "timestamp" : "2020-06-10T14:55:25.534Z",
  "level" : "INFO",
  "thread" : "Catalina-utility-1",
  "logger" : "org.springframework.web.servlet.DispatcherServlet",
  "message" : "FrameworkServlet 'dispatcher': initialization completed in 72 ms",
  "context" : "default"
}

What am I doing wrong?

SOLUTION

I was using the wrong libraries for my needs:

• logback-jackson
• logback-json-classic

Because of the fact that I need to process logs through Logstash I've corrected my configuration like this:

pom.xml

<dependency>
                <groupId>ch.qos.logback</groupId>
                <artifactId>logback-classic</artifactId>
                <version>1.2.3</version>
            </dependency>
            <dependency>
                <groupId>net.logstash.logback</groupId>
                <artifactId>logstash-logback-encoder</artifactId>
                <version>6.4</version>
            </dependency>

logback.xml

<appender name="Console" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LogstashEncoder">
            <customFields>{"customer":"X", "appname":"Y", "environment":"dev"}</customFields>            
        </encoder>    
    </appender>

and now It works fine.

Answer 1

I just stumbled this question because I had the same problem, and I found a solution, with logback-jackson and logback-json-classic.

Option 1: Per-Thread via Mapped Diagnostic Context (MDC)

SLF4j's Mapped Diagnostic Context is a per-thread key-value store that we can use to write custom structured data to the log output.

MDC.put("customKey", "customValue");

Logback's JsonLayout will automatically print this value under a special mdc JSON object without any further configuration.

{ [...], "mdc": {"customKey", "customValue"}} 

Note that the MDC is constructed per thread and if it is empty, no mdc field is printed to the log output.

Option 2: Global (for all threads)

If you want custom fields to appear at the JSON output's root, you need to create a custom, but simple Layout class that extends JsonLayout. JsonLayout provides us with a addCustomDataToJsonMap we can override.

package com.mypackage;

import ch.qos.logback.contrib.json.classic.JsonLayout;

public class CustomJsonLayout extends JsonLayout {

  @Override
  protected void addCustomDataToJsonMap(Map<String, Object> map, ILoggingEvent event) {
    map.put("customKey", "customValue");
  }
}

Now, you just need to tell Logback to use CustomJsonLayout instead of JsonLayout in your logback.xml file and keep the rest the same.

<layout class="com.mypackage.CustomJsonLayout">
    ...
</layout>

Now, any log message will have the following output:

{ ..., "customKey": "customValue"}