GAE - Python 3.7 - How to log?

Question

I have a google app engine project in python 3.7 in which I would like to write some logs. I am used to program in app engine python 2.7 and I was using the simple code:

 logging.info('hi there!')

to write any log onto the google cloud log console. That command above now doesn't work anymore and it says:

logging has no attribute 'info'

I searched and I found this possible new code

from flask import Flask
from google.cloud import logging

app = Flask(__name__)

@app.route('/l')
def hello():
    logging_client = logging.Client()
    log_name = LOG_NAME
    logger = logging_client.logger(LOG_NAME)
    text = 'Hello, world!'
    logger.log_text(text, severity='CRITICAL')
    return text

This code above doesn't give any error in the stack-driver report page BUT it displays nothing at all in the log page.

So how can I write a log for my app engine project in python3.7?

Answer 1

The second generation standard environment (which includes python 3.7) is IMHO closer to the flexible environment than to the first generation standard environment (which includes python 2.7).

Many of the APIs which had customized versions for the 1st generation (maintained by the GAE team) were not (or at least not yet) ported in the 2nd generation if the respective functionality was more or less covered by alternate, more generic approaches, already in use in the flexible environment (most of them based on services developed and maintained by teams other than the GAE one).

You'll notice the similarity between many of the service sections in these 2 migration guides (which led me to the above summary conclusion):

• Understanding differences between Python 2 and Python 3 on the App Engine standard environment
• Migrating Services from the Standard Environment to the Flexible Environment (i.e 1st generation standard to flexible)

Logging is one of the services listed in both guides. The 1st generation used a customized version of the standard python logging library (that was before Stackdriver became a standalone service). For the 2nd generation logging was simply delegated to using the now generally available Stackdriver logging service (which is what the snippet you shown comes from). From Logging (in the 1st guide):

Request logs are no longer automatically correlated but will still appear in Stackdriver Logging. Use the Stackdriver Logging client libraries to implement your desired logging behavior.

The code snippet you show corresponds, indeed, to Stackdriver Logging. But you seem to be Using the client library directly. I don't know if this is a problem (GAE is often a bit different), but maybe you could also try using the standard Python logging instead:

• Connecting the library to Python logging:

To send all log entries to Stackdriver by attaching the Stackdriver Logging handler to the Python root logger, use the setup_logging helper method:

# Imports the Google Cloud client library
import google.cloud.logging

# Instantiates a client
client = google.cloud.logging.Client()

# Connects the logger to the root logging handler; by default this captures
# all logs at INFO level and higher
client.setup_logging()

• Using the Python root logger:

Once the handler is attached, any logs at, by default, INFO level or higher which are emitted in your application will be sent to Stackdriver Logging:

# Imports Python standard library logging
import logging

# The data to log
text = 'Hello, world!'

# Emits the data using the standard logging module
logging.warn(text)

There are some GAE-specific notes in there as well (but I'm not sure if they also cover the 2nd generation standard env):

Google App Engine grants the Logs Writer role by default.

The Stackdriver Logging library for Python can be used without needing to explicitly provide credentials.

Stackdriver Logging is automatically enabled for App Engine applications. No additional setup is required.

Note: Logs written to stdout and stderr are automatically sent to Stackdriver Logging for you, without needing to use Stackdriver Logging library for Python.

Maybe worth noting that the viewing the logs would likely be different as well outside the 1st generation standard env (where app logs would be neatly correlated to the request logs).

And there's also the Using Stackdriver Logging in App Engine apps guide. It doesn't specifically mention the 2nd generation standard env (so it might need an update) but has good hints for the flexible environment which might be useful. For example the Linking app logs and requests section might be of interest if the missing request logs correlation has anything to do with it.

Answer 2

Even though logging works differently in Python 2.7 and 3.7, the same method of logging provided in Reading and Writing Application Logs in Python 2.7 should work for Python 3.7 as well since logs written to stdout and stderr will still appear in the Stackdriver Logging.

Import logging

logging.getLogger().setLevel(logging.DEBUG)
logging.debug('This is a debug message')

logging.getLogger().setLevel(logging.INFO)
logging.info('This is an info message')
logging.warning('This is a warning message')
logging.error('This is an error message')
logging.critical('This is a critical message')
#logging.warn is deprecated
logging.warn('This is a warning' message)

=======================================

Import logging

app.logger.setLevel(logging.ERROR)
app.logger.error('This is an error message')

However, log entries are no longer automatically correlated with requests as in Python 2.7, which is why you see them in plain text. I have created a feature request to address this which you can follow here.