Sending data to CloudWatch using the AWS-SDK

Question

I want to write data to CloudWatch using the AWS-SDK (or whatever may work).

I see this:

The only method that looks remotely like publishing data to CloudWatch is the putMetricData method..but it's hard to find an example of using this.

Does anyone know how to publish data to CloudWatch?

When I call this:

cw.putMetricData({
  Namespace: 'ec2-memory-usage',
  MetricData: [{
    MetricName:'first',
    Timestamp: new Date()
  }]
}, (err, result) => {
  console.log({err, result});
});

I get this error:

{ err:
   { InvalidParameterCombination: At least one of the parameters must be specified.
       at Request.extractError (/Users/alex/codes/interos/jenkins-jobs/jobs/check-memory-ec2-instances/node_modules/aws-sdk/lib/protocol/query.js:50:29)
       at Request.callListeners (/Users/alex/codes/interos/jenkins-jobs/jobs/check-memory-ec2-instances/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
       at Request.emit (/Users/alex/codes/interos/jenkins-jobs/jobs/check-memory-ec2-instances/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
       at Request.emit (/Users/alex/codes/interos/jenkins-jobs/jobs/check-memory-ec2-instances/node_modules/aws-sdk/lib/request.js:683:14)
       at Request.transition (/Users/alex/codes/interos/jenkins-jobs/jobs/check-memory-ec2-instances/node_modules/aws-sdk/lib/request.js:22:10)
       at AcceptorStateMachine.runTo (/Users/alex/codes/interos/jenkins-jobs/jobs/check-memory-ec2-instances/node_modules/aws-sdk/lib/state_machine.js:14:12)
       at /Users/alex/codes/interos/jenkins-jobs/jobs/check-memory-ec2-instances/node_modules/aws-sdk/lib/state_machine.js:26:10
       at Request.<anonymous> (/Users/alex/codes/interos/jenkins-jobs/jobs/check-memory-ec2-instances/node_modules/aws-sdk/lib/request.js:38:9)
       at Request.<anonymous> (/Users/alex/codes/interos/jenkins-jobs/jobs/check-memory-ec2-instances/node_modules/aws-sdk/lib/request.js:685:12)
       at Request.callListeners (/Users/alex/codes/interos/jenkins-jobs/jobs/check-memory-ec2-instances/node_modules/aws-sdk/lib/sequential_executor.js:116:18)
     message: 'At least one of the parameters must be specified.',
     code: 'InvalidParameterCombination',
     time: 2019-07-08T19:41:41.191Z,
     requestId: '688a4ff3-a1b8-11e9-967e-431915ff0070',
     statusCode: 400,
     retryable: false,
     retryDelay: 7.89360948163893 },
  result: null }

Answer 1

You're getting this error because you're not specifying any metric data. You're only setting the metric name and the timestamp. You also need to send some values for the metric.

Let's say your application is measuring the latency of requests and you observed 5 requests, with latencies 100ms, 500ms, 200ms, 200ms and 400ms. You have few options for getting this data into CloudWatch (hence the At least one of the parameters must be specified. error).

1. You can publish these 5 values one at a time by setting the Value within the metric data object. This is the simplest way to do it. CloudWatch does all the aggregation for you and you get percentiles on your metrics. I would not recommended this approach if you need to publish many observations. This option will result in the most requests made to CloudWatch, which may result in a big bill or throttling from CloudWatch side if you start publishing too many observations.

For example:

MetricData: [{
    MetricName:'first',
    Timestamp: new Date(),
    Value: 100
 }]

2. You can aggregate the data yourself and construct and publish the StatisticValues. This is more complex on your end, but results in the fewest requests to CloudWatch. You can aggregate for a minute for example and execute 1 put per metric every minute. This will not give you percentiles (since you're aggregating data on your end, CloudWatch doesn't know the exact values you observed). I would recommend this if you do not need percentiles.

For example:

MetricData: [{
    MetricName:'first',
    Timestamp: new Date(),
    StatisticValues: {
        Maximum: 500,
        Minimum: 100,
        SampleCount: 5,
        Sum: 1400
      }
 }]

3. You can count the observations and publish Values and Counts. This is kinda the best of both worlds. There is some complexity on your end, but counting is arguably easier than aggregating into StatisticValues. You're still sending every observation so CloudWatch will do the aggregation for you, so you'll get percentiles. The format also allows more data to be sent than in the option 1. I would recommend this if you need percentiles.

For example:

MetricData: [{
    MetricName:'first',
    Timestamp: new Date(),
    Values: [100, 200, 400, 500],
    Counts: [1, 2, 1, 1]
 }]

See here for more details for each option: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/CloudWatch.html#putMetricData-property