Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to search Datadog logs by Attribute

Tags:

datadog

Question about searching logs in Datadog.

Search works on regular strings in the CONTENT portion of the log. However, if JSON is passed to the CONTENT portion, the JSON elements are automatically parsed into Attributes. But the Attributes are NOT searchable.

How do I search for logs by Attribute?

It seems like a step backwards to supply log data in JSON to improve indexing, but then LOSE the ability to search on those elements.

like image 258
Pete Lunenfeld Avatar asked Aug 29 '20 18:08

Pete Lunenfeld


People also ask

What is the effect of using Datadog's patterns view in the log Explorer?

The Log Patterns view can help you quickly see the big picture when you're flooded with verbose application logs, but it also allows you to swiftly drill down to get more details. You can click on any cluster to see individual log entries that exhibit that pattern.

Does Datadog collect logs?

Datadog Log Management, also referred to as Datadog logs or logging, removes these limitations by decoupling log ingestion from indexing. This enables you to cost-effectively collect, process, archive, explore, and monitor all of your logs without limitations, also known as Logging without Limits*.

How long are logs retained in Datadog?

Instead of retaining and sifting through a large number of logs, you can create a single metric to track the trends those logs reflect. Datadog retains that metric at full granularity for 15 months.

Can you export data from Datadog?

You can access the information you copied from the bottom of your screen as you navigate to other pages, and export items from the Clipboard to new or existing incidents, Notebooks, and dashboards.


2 Answers

You need to tell Datadog that you're interested in that content by creating a facet from the field. Click a log message, mouse over the attribute name, click the gear on the left, then Create facet for @...

For logs indexed after you create the facet, you can search with @fieldName:text*, where fieldName is the name of your field. You'll need to re-hydrate (reprocess) earlier logs to make them searchable.

You won't need to create a facet if you use fields from the standard attributes list.

like image 174
kielni Avatar answered Sep 20 '22 13:09

kielni


The error message itself is not a good fit to be defined as a facet.

If you are using JSON and want the main message (say from a msg json field) to be searchable in the Datadog content field. Instead of making facet for msg, you can define a "Message Remapper" in the log configuration to map it to the Content. And then you can do wildcard searches.

log config screenshot

like image 35
sanjeev Avatar answered Sep 21 '22 13:09

sanjeev