Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Search AWS CloudWatch Logs with special character in JSON property name

We use AWS CloudWatch Logs to debug our services running on AWS, and a number of the property names in our logs use colons as delimiters, but we can't figure out how to search using these property names.

Here's an example of a log:

{
  "Counts": {
    "RouteHandler:GetCookies": {
      "value": 1
    }
  }
}

Using the console, I've tried a query like this:

{ $.Counts.RouteHandler:GetCookies.value = 1 }

Of course, special characters often have specific uses in query languages and otherwise, so I've try to escape it somehow.

{ $.Counts.RouteHandler\:GetCookies.value = 1 }
// JavaScript inspired
{ $.Counts["RouteHandler:GetCookies"].value = 1 }
// Special character removed
{ $.Counts.RouteHandlerGetCookies.value = 1 }

Nothing I could come up with worked, and maybe it's just not possible. The docs don't seem to address this type of scenario.

Anyone know how to search JSON logs where properties contain special characters, or know definitively whether or not this is supported?

In new work, we're going to use a different delimiter, but we're not going to go back and change it everywhere.

like image 894
EmptyArsenal Avatar asked Mar 27 '17 21:03

EmptyArsenal


People also ask

How do I search AWS CloudWatch?

CloudWatch search expressions: Using math expressions You can use a search expression within a math expressions in a graph. For example, SUM(SEARCH(' {AWS/Lambda, FunctionName} MetricName="Errors" ', 'Sum', 300)) returns the sum of the Errors metric of all your Lambda functions.

Are CloudWatch logs JSON?

We are happy to announce support for monitoring JSON-formatted logs with CloudWatch Logs. This capability enables you to create graphs and receive notifications when your JSON-formatted log events contain terms or match conditions that you choose.


1 Answers

At the time of writing, this is not possible. AWS will probably fix that at some point, but for now the only workaround would be to use the non-JSON syntax and search for the exact string. The following filter:

"\"RouteHandler:GetCookies\": {\"value\": 1}"

will match this log event:

{"Counts": {"RouteHandler:GetCookies": {"value": 1}}}

Obviously the downside is that whitespace and position matter.

like image 121
Daniel Vassallo Avatar answered Sep 21 '22 11:09

Daniel Vassallo