Hot partition problem in DynamoDB gone with the new on-demand feature?

Question

I read the following announcement with great interest.

https://aws.amazon.com/about-aws/whats-new/2018/11/announcing-amazon-dynamodb-on-demand/

The new "on-demand" feature really helps with capacity planning. Reading the documentation, I can't really see if they do some "magic" to resolve the problem of hot partitions, and partition key distribution.

Is partition key design just as important if you provision a table "on-demand"?

Answer 1

Yes, partition key design is just as important. That aspect has not changed.

Since you mentioned adaptive capacity in a comment, one thing to make sure is clear. Once it is on for a table, it is on and DynamoDB is monitoring your table.

Answer 2

There are two features at play here: * On-demand capacity mode * Adaptive capacity

On-demand capacity mode allows you to pay per each request to DynamoDB instead of provisioning a particular amount of RCUs/WCUs (this is called provisioned capacity). The benefit is that you only pay for what you use (and not for what you provision), but the downside is that if you receive a constant flow of requests, you would end up paying more if you provisioned the right amount of RCUs/WCUs. The on-demand capacity mode is the best suit for spiky traffic, while the provisioned mode is better for applications with a constant, predictable stream of requests

Adaptive capacity is a different feature, and it can work with either on-demand or provisioned capacity modes. It allows to "borrow" unused capacity from other partitions if one of your partitions receive a higher share of requests. It used to take some time to enable adaptive capacity, but as for now, adaptive capacity is enabled immediately.

Even with adaptive capacity, a good key design is still important. It only helps with cases when it is hard to achieve a balanced distribution of requests among shards. A single partition in DynamoDB can only handle up to 3K RCUs and 1K WCUs. So if a single partition receives more than that even with adaptive capacity requests will be throttled. So you have to design your keys to avoid this scenario.