DynamoDB Throttling

Question

I have a dynamo table with 5 read and 5 write capacity. In this table I had two records.

I then decided to see what kind of error response I would get by pushing a large number of writes through at once (roughly 4000 records in one burst). The blue 'consumed' line shot straight over the red 'capacity' line however I got no error message of any kind.

The metrics show throttling taking place but my reads still occur and my writes still occur if I go over both capacity levels.

I have spent over 30 minutes pushing the read and writes well above capacity, with no error yet.

I'm not sure if this is because I am using the official javascript SDK in node js, which maybe transparently handles throttling and retries the throttled queries?

I'm hoping somebody can give me some guidance on this.

Thanks

Answer 1

Just wanted to add a notice about throttling to the answer by @Luc Hendriks

Even after you are out of 'burst capacity' and DynamoDb starts throttling, you will get ProvisionedThroughputExceededException relatively rarely (as per my experience). This is because SDK silently retries throttled requests (so you were right, SDK transparently handles throttling).

You can use maxRetries parameter to disable automatic retries, so that you will get ProvisionedThroughputExceededException right after you see throttling in metrics.

Here is an example how to perform an update request without automatic retries:

var aws_dynamodb = new aws.DynamoDB({maxRetries: 0}),
    aws_dynamodb_doc = new aws.DynamoDB.DocumentClient({service: aws_dynamodb});

// ...
aws_dynamodb_doc.update(params, function(err, data) {
  // ...
});

Answer 2

This is because of burst capacity. You can read more about it on the AWS docs: http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GuidelinesForTables.html#GuidelinesForTables.Bursting

Basically what it means is you can have up to 300 seconds worth of 'stored capacity' and use that as a pool when you have a spike in load. In your case that would be a pool of 1500 requests you can use instantly and is refilled again when capacity is unused.