How can I visualize timeseries data aggregated by more than one dimension on AWS insights?

Question

I'd like to use cloudwatch insights to visualize a multiline graph of average latency by host over time. One line for each host.

This stats query extracts the latency and aggregates it in 10 minute buckets by host, but it doesn't generate any visualization.

stats avg(latencyMS) by bin(10m), host

bin(10m) | host | avg(latencyMS)
0m       |    1 |           120
0m       |    2 |           220
10m      |    1 |           130
10m      |    2 |           230

The docs call this out as a common mistake but don't offer any alternative.

The following query does not generate a visualization, because it contains more than one grouping field.

stats avg(myfield1) by bin(5m), myfield4

aws docs

Experementally, cloudwatch will generate a multi line graph if each record has multiple keys. A query that would generate a line graph must return results like this:

bin(10m) | host-1 avg(latencyMS) | host-2 avg(latencyMS)
0m       |    120                |                   220
10m      |    130                |                   230

I don't know how to write a query that would output that.

Answer 1

Parse individual message for each host then compute their stats.

For example, to get average latency for responses from processes with PID=11 and PID=13.

parse @message /\[PID:11\].* duration=(?<pid_11_latency>\S+)/
| parse @message /\[PID:13\].* duration=(?<pid_13_latency>\S+)/
| display @timestamp, pid_11_latency, pid_13_latency
| stats avg(pid_11_latency), avg(pid_13_latency) by bin(10m)
| sort @timestamp desc
| limit 20

The regular expressions extracts duration for processes having id 11 and 13 to parameters pid_11_latency and pid_13_latency respectively and fills null where there is no match series-wise.

You can build from this example by creating the match regular expression that extracts for metrics from message for hosts you care about.