How do I map my java app logging events to corresponding cloud logging event levels in GCP Felexible non-compat App Engine?

Question

I am new to GCP AppEngine and I chose the Flexible environment for several reasons. However, I am shocked to find out that the flexible environment's non-"compatible" runtimes appear to not allow me to map my app's logging events to the appropriate log levels in cloud logging. Am I reading this correctly? https://cloud.google.com/appengine/docs/flexible/java/writing-application-logs#writing_application_logs_1

And this page was really unhelpful. https://cloud.google.com/java/getting-started/logging-application-events

This is after several hours of reading GAE logging woes and trying to determine which applied to the Standard environment vs. Flexible. Best I can tell, event level mapping is possible in the standard environment.

However, for more fine-grained control of the log level display in the Cloud Platform Console, the logging framework must use a java.util.logging adapter. https://cloud.google.com/appengine/docs/java/how-requests-are-handled#Java_Logging

OK. That's a vague reference, but I think I saw something more clear somewhere else.

Regardless, shouldn't this be easier in the "flexible" environment? Who doesn't want to easily filter events by Logging levels?

Update: I clarified the question to indicate that I am asking about the non-compatible runtimes in the GAE flexible environment.

Answer 1

Here is how I got cloud logging to work using SLF4J. This works on a non-compatible Java GAE Flex environment.

logback.xml

<configuration debug="true">
    <appender name="FILE" class="ch.qos.logback.core.FileAppender">
        <file>/var/log/app_engine/custom_logs/app.log.json</file>
        <append>true</append>
        <!-- encoders are assigned the type ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
        <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
            <layout
                class="putyourpackagenamehere.GCPCloudLoggingJSONLayout">
                <pattern>%-4relative [%thread] %-5level %logger{35} - %msg</pattern>
            </layout>
        </encoder>
    </appender>
    <root level="DEBUG">
        <appender-ref ref="FILE" />
    </root>
</configuration>

Here is the PatternLayout class I used to produce the JSON on a single line in the log file.

import static ch.qos.logback.classic.Level.DEBUG_INT;
import static ch.qos.logback.classic.Level.ERROR_INT;
import static ch.qos.logback.classic.Level.INFO_INT;
import static ch.qos.logback.classic.Level.TRACE_INT;
import static ch.qos.logback.classic.Level.WARN_INT;

import java.util.Map;

import org.json.JSONObject;

import com.homedepot.ta.wh.common.logging.GCPCloudLoggingJSONLayout.GCPCloudLoggingEvent.GCPCloudLoggingTimestamp;

import ch.qos.logback.classic.Level;
import ch.qos.logback.classic.PatternLayout;
import ch.qos.logback.classic.spi.ILoggingEvent;

/**
 * Format a LoggingEvent as a single line JSON object  
 * 
 *  <br>https://cloud.google.com/appengine/docs/flexible/java/writing-application-logs
 *  
 *  <br>From https://cloud.google.com/appengine/articles/logging
 *  <quote>
 *  Applications using the flexible environment should write custom log files to the VM's log directory at 
 *  /var/log/app_engine/custom_logs. These files are automatically collected and made available in the Logs Viewer. 
 *  Custom log files must have the suffix .log or .log.json. If the suffix is .log.json, the logs must be in JSON 
 *  format with one JSON object per line. If the suffix is .log, log entries are treated as plain text.
 *  </quote>
 *  
 *  Nathan: I can't find a reference to this format on the google pages but I do remember getting the format from some
 *  GO code that a googler on the community slack channel referred me to.   
 */
public class GCPCloudLoggingJSONLayout extends PatternLayout {

    @Override
    public String doLayout(ILoggingEvent event) {
        String formattedMessage = super.doLayout(event);
        return doLayout_internal(formattedMessage, event);
    }

    /* for testing without having to deal wth the complexity of super.doLayout() 
     * Uses formattedMessage instead of event.getMessage() */
    String doLayout_internal(String formattedMessage, ILoggingEvent event) {
        GCPCloudLoggingEvent gcpLogEvent = new GCPCloudLoggingEvent(formattedMessage
                                                                    , convertTimestampToGCPLogTimestamp(event.getTimeStamp())
                                                                    , mapLevelToGCPLevel(event.getLevel())
                                                                    , null);
        JSONObject jsonObj = new JSONObject(gcpLogEvent);
        /* Add a newline so that each JSON log entry is on its own line.
         * Note that it is also important that the JSON log entry does not span multiple lines.
         */
        return jsonObj.toString() + "\n"; 
    }

    static GCPCloudLoggingTimestamp convertTimestampToGCPLogTimestamp(long millisSinceEpoch) {
        int nanos = ((int) (millisSinceEpoch % 1000)) * 1_000_000; // strip out just the milliseconds and convert to nanoseconds
        long seconds = millisSinceEpoch / 1000L; // remove the milliseconds
        return new GCPCloudLoggingTimestamp(seconds, nanos);
    }

    static String mapLevelToGCPLevel(Level level) {
        switch (level.toInt()) {
        case TRACE_INT:
            return "TRACE";
        case DEBUG_INT:
            return "DEBUG";
        case INFO_INT:
            return "INFO";
        case WARN_INT:
            return "WARN";
        case ERROR_INT:
            return "ERROR";
        default:
            return null; /* This should map to no level in GCP Cloud Logging */
        }
    }

    /* Must be public for JSON marshalling logic */
    public static class GCPCloudLoggingEvent {
        private String message;
        private GCPCloudLoggingTimestamp timestamp;
        private String traceId;
        private String severity;

        public GCPCloudLoggingEvent(String message, GCPCloudLoggingTimestamp timestamp, String severity,
                String traceId) {
            super();
            this.message = message;
            this.timestamp = timestamp;
            this.traceId = traceId;
            this.severity = severity;
        }

        public String getMessage() {
            return message;
        }

        public void setMessage(String message) {
            this.message = message;
        }

        public GCPCloudLoggingTimestamp getTimestamp() {
            return timestamp;
        }

        public void setTimestamp(GCPCloudLoggingTimestamp timestamp) {
            this.timestamp = timestamp;
        }

        public String getTraceId() {
            return traceId;
        }

        public void setTraceId(String traceId) {
            this.traceId = traceId;
        }

        public String getSeverity() {
            return severity;
        }

        public void setSeverity(String severity) {
            this.severity = severity;
        }

        /* Must be public for JSON marshalling logic */
        public static class GCPCloudLoggingTimestamp {
            private long seconds;
            private int nanos;

            public GCPCloudLoggingTimestamp(long seconds, int nanos) {
                super();
                this.seconds = seconds;
                this.nanos = nanos;
            }

            public long getSeconds() {
                return seconds;
            }

            public void setSeconds(long seconds) {
                this.seconds = seconds;
            }

            public int getNanos() {
                return nanos;
            }

            public void setNanos(int nanos) {
                this.nanos = nanos;
            }

        }       
    }

    @Override
    public Map<String, String> getDefaultConverterMap() {
        return PatternLayout.defaultConverterMap;
    }   
}

Answer 2

The log levels provided by java.util.logging will map to the appropriate log levels in Cloud Logging. Logging on Flexible runtimes essentially works the same as it does on Standard.

Edit: It seems the rationale for the 'Writing Application Logs' page is that the Cloud Logging mappings don't work for all of the runtimes. However, they do seem to currently work for at least the '-compat' runtimes and Java custom runtimes. Workarounds for others are provided elsewhere in the docs (see below):

The recommended default method of logging in a Java application is to use java.util.logging (for Python it's the 'logging' module, and for Go it's the 'log' package, all of which provide log levels which map to Cloud Logging levels). I'll request that these pages get updated.

The other documents you linked to provide accurate information about logging for Java. Regarding the section you quoted, the full paragraph it was pulled from provides context. It's saying any logging framework that writes to stderr or stdout will work, but it needs to use 'java.util.logging' if you want more fine-grained log levels other than 'INFO' or 'WARNING'. A full code sample for using 'java.util.logging' is provided directly underneath the quoted section, and others are provided on the other document you mentioned, 'Logging Application Events with Java'.

Update: The 'Getting Started' guides contain specific details on how to configure logging for each runtime:

Java
https://cloud.google.com/java/getting-started/logging-application-events#understanding_the_code

Python
https://cloud.google.com/python/getting-started/logging-application-events#understanding_the_code

Go
https://cloud.google.com/go/getting-started/logging-application-events

NodeJS
https://cloud.google.com/nodejs/getting-started/logging-application-events#understanding_the_code

Ruby
https://cloud.google.com/ruby/getting-started/logging-application-events#application_structure

PHP
https://cloud.google.com/php/getting-started/logging-application-events