Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

AWS CloudWatch log subscription filters decode

I am using CloudWatch log subscription filters stream to Lambda and publish a message to an SNS topic. But it will output garbled message and can't success decode.

my output:

k
%"
 jVbB

If not decode will output like this:

{ "awslogs": {"data": "BASE64ENCODED_GZIP_COMPRESSED_DATA"} }

My code is below and it is using nodejs:

console.log("Loading function");
var AWS = require("aws-sdk");

exports.handler = function(event, context) {
    var eventText = JSON.stringify(event, null, 2);
    var decodeText = new Buffer(eventText, 'base64').toString('ascii');
    console.log("Received event:", eventText);
    var sns = new AWS.SNS();
    var params = {
        Message: decodeText, 
        Subject: "Test SNS From Lambda",
        TopicArn: "arn:aws:sns:region:account:snsTopic"
    };
    sns.publish(params, context.done);
};
like image 443
Kelvin Avatar asked May 14 '18 09:05

Kelvin


People also ask

How do you parse in CloudWatch logs?

Use the sort command to display log events in ascending ( asc ) or descending ( desc ) order. Use the limit command to specify the number of log events that you want your query to return. Use the parse command to extract data from a log field and create an ephemeral field that you can process in your query.

What is a subscription filter?

A subscription filter defines the filter pattern to use for filtering which log events get delivered to your AWS resource, as well as information about where to send matching log events to. Each log group can have up to two subscription filters associated with it.


Video Answer


1 Answers

CloudWatch Logs are delivered to the subscribed Lambda function as a list that is gzip-compressed and base64-encoded.

Here is an example of how to decode and unzip the list of logs:

const zlib = require('zlib');

exports.handler = async (event, context) => {
  if (event.awslogs && event.awslogs.data) {
    const payload = Buffer.from(event.awslogs.data, 'base64');

    const logevents = JSON.parse(zlib.unzipSync(payload).toString()).logEvents;

    for (const logevent of logevents) {
      const log = JSON.parse(logevent.message);
      console.log(log);
    }
  }
};
like image 111
jarmod Avatar answered Oct 18 '22 01:10

jarmod