Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using regular expression filter as aws cloudwatch logs metric filter

For logs sent to AWS cloudwatch-logs, I want to create metric filter separating a numeric field from the log matching pattern. So as per documentation there is [w1=word_pattern1||word_pattern2, w2, ...] like expression to get fields (based on space separator/delimiter). But my logs have some other separators say like : or ,. So is there any way of parsing such fields using some regex pattern.

sample log pattern: date-time:action:filename:status

e.g.

2018-11-02 12:23:00:delete-file:filename-20180709:done

or like

2018-11-02 12:23:00, deleting-file filename-20180709 done

I can parse the log line for file code with grok regex like:

(?<date>[\w-]+ [\w:]+), (?<action>\w+-\w+) filename-(?<file-code>\d{8}) \w+

but I'm not able to find any equivalent for the metric filter. Is it possible?

like image 972
v_sukt Avatar asked Feb 11 '19 08:02

v_sukt


People also ask

How do you make a CloudWatch metric filter?

To create a metric filter using the CloudWatch consoleOpen the CloudWatch console at https://console.amazonaws.cn/cloudwatch/ . In the navigation pane, choose Logs, and then choose Log groups. Choose the name of the log group. Choose Actions , and then choose Create metric filter.

What is CloudWatch log metric filter?

Metric filters define the terms and patterns to look for in log data as it is sent to CloudWatch Logs. CloudWatch Logs uses these metric filters to turn log data into numerical CloudWatch metrics that you can graph or set an alarm on.

How do I automate CloudWatch logs?

To send action output to CloudWatch Logs (console)Open the AWS Systems Manager console at https://console.aws.amazon.com/systems-manager/ . In the navigation pane, choose Automation. Choose the Preferences tab, and then choose Edit. Select the check box next to Send output to CloudWatch Logs.


1 Answers

I used to work in the team that built CloudWatch Logs. At the time of writing, regular expressions are not supported in metric filters.

like image 128
Daniel Vassallo Avatar answered Sep 20 '22 09:09

Daniel Vassallo