How to configure FastAPI to publish logs to CloudWatch?

Question

I have a FastAPI service that works as expected in every regard except the logging, only when it runs as a AWS Lambda function.

When running it locally the logs are displayed on the console as expected:

INFO:     127.0.0.1:62160 - "POST /api/v1/feature-requests/febbbc21-9650-44e6-8df5-80c8bb33b6ea/upvote HTTP/1.1" 200 OK
INFO:     127.0.0.1:62158 - "OPTIONS /api/v1/feature-requests HTTP/1.1" 200 OK
INFO:     127.0.0.1:62160 - "GET /api/v1/feature-requests HTTP/1.1" 200 OK
INFO:     127.0.0.1:62158 - "OPTIONS /api/v1/feature-requests-meta HTTP/1.1" 200 OK
INFO:     127.0.0.1:62160 - "GET /api/v1/feature-requests-meta HTTP/1.1" 200 OK
INFO:     127.0.0.1:62160 - "GET /api/v1/feature-requests/febbbc21-9650-44e6-8df5-80c8bb33b6ea HTTP/1.1" 200 OK
INFO:     127.0.0.1:62160 - "GET /api/v1/feature-requests-meta/febbbc21-9650-44e6-8df5-80c8bb33b6ea HTTP/1.1" 200 OK

However, when deployed as a Lambda function the logs are not there:

2022-09-07T10:44:57.426+02:00   START RequestId: fd44ae47-5bfb-42e3-aeb4-d9f29857bb39 Version: $LATEST
2022-09-07T10:44:57.604+02:00   END RequestId: fd44ae47-5bfb-42e3-aeb4-d9f29857bb39
2022-09-07T10:44:57.604+02:00   REPORT RequestId: fd44ae47-5bfb-42e3-aeb4-d9f29857bb39 Duration: 177.85 ms Billed Duration: 178 ms Memory Size: 2048 MB Max Memory Used: 152 MB Init Duration: 1733.88 ms
2022-09-07T10:45:00.299+02:00   START RequestId: 08a7a6da-c2c6-446c-baa3-1d08c9816f5b Version: $LATEST
2022-09-07T10:45:00.318+02:00   END RequestId: 08a7a6da-c2c6-446c-baa3-1d08c9816f5b

Even for the logs that are produced by our code (as opposed to the framework) are not visible when running as a Lambda function.

Configuration:

In app.py

LOG = logging.getLogger()
log_format = "%(asctime)s %(levelname)s %(message)s"
log_date_fmt = "%Y-%m-%d %H:%M:%S"
logging.basicConfig(
    format=log_format,
    level=logging.INFO,
    datefmt=log_date_fmt,
)

In every other Python file:

LOG = logging.getLogger(__name__)

logging.conf

[loggers]
keys=root,api,config

[handlers]
keys=console_handler

[formatters]
keys=normal_formatter

[logger_root]
level=INFO
handlers=console_handler

[logger_api]
level=INFO
handlers=console_handler
qualname=api
propagate=0

[logger_config]
level=INFO
handlers=console_handler
qualname=config
propagate=0

[handler_console_handler]
class=StreamHandler
level=INFO
formatter=normal_formatter
args=(sys.stdout,)

[formatter_normal_formatter]
format=%(asctime)s %(levelname)s %(name)s %(message)s
datefmt=%Y-%m-%d %H:%M:%S

I am not sure what else needs to happen to get the logs in CloudWatch.

Answer 1

try cloudwatch library

Run pip install cloudwatch in the console

then in your code:

import logging
from cloudwatch import cloudwatch

logger = logging.getLogger('cloudwatch_logger')
formatter = logging.Formatter('%(asctime)s : %(levelname)s - %(message)s')
handler = cloudwatch.CloudwatchHandler(log_group = 'cloudwatch_log_group')
handler.setFormatter(formatter)
logger.setLevel(logging.DEBUG)
logger.addHandler(handler)

and then either use this logger to log to CloudWatch logger.warning('I am here') or add the handler to the root logger via logging configuration