Cloudwatch Insights search in multiline logs

Question

Example log:

REQUEST-ID:123 Video id=444
REQUEST-ID:123 Request=error
REQUEST-ID:534 Video id=555
REQUEST-ID:534 Request=ok

Question, how to find Video id of all requests with an error?

Answer 1

Here is the official response from AWS support:

Currently, we do not have support for nested queries, and hence your use-case won't be achievable with Insights. However, we can confirm that we have a feature request existing at our end for this use-case. We don't have an ETA now on when this will be implemented. I hope you understand. Please keep a watch on below link for any new feature updates:

https://aws.amazon.com/new/

As a workaround, for now, you could download partial query results(Run the query -> Actions -> Export -> Download query results(CSV) and filter the results to identify the video ids. Please find below one sample query that can help you in this case:

fields @timestamp, @message
| parse @message 'REQUEST-ID:* Request=*' as REQUESTID,Request
| parse @message 'REQUEST-ID:* Video id=*' as REQUESTIDVID,Videoid
| DISPLAY REQUESTID,Request,REQUESTIDVID,Videoid